A Longitudinal Assessment of Written Communication among Students at a Private University in Kenya

Regular assessment of program outcomes is crucial for efficient curriculum review and improvement. Most institutions especially in Africa lack frameworks for assessing the effectiveness of the programs they offer. For those that do, they involve end of course assessment which does not allow early interventions before students graduate. This paper highlights the importance of using a longitudinal approach to assess program outcome. The paper presents findings of a longitudinal assessment of students’ writing skills at United States International University (USIU-A). The main objective of the study was to assess progress made by students in written communication competence right from entry to their final level of the English courses offered at the institution. The study used an analytical rubric to assess students written scripts at distinct levels from entry to their final level of the English courses. This was aimed at providing instructional relevant feedback which could be used to inform curriculum review and classroom instruction. The results show general improvement in the students writing skills over their four English course levels. The study also showed that the area where students struggle the most is Syntax and Grammar. Recommendations made from the findings include providing students with a language intensive program before they begin their degree program, promoting writing across the curriculum approach, and limiting language class sizes to 30 for effective teaching, evaluation, and feedback.


I. INTRODUCTION
Program assessment in higher education is important.With changing market trends across the globe, it is important that learning institutions align their programs with market demands.This can be achieved through regular program assessment and review.Analyzing student learning at program-level, assists faculty and administrators gain an understanding of whether the courses offered are meeting the program learning outcomes and whether instructional methods are effective in delivering the programs (Council of Graduate Schools, 2011).Program learning outcomes (PLO) assessment is a form of action research in the sense that the results are used to inform decisions about the allocation of resources, instructional approaches, staffing, and other policies and practices aimed at program improvement (Kuh & Ewell, 2010;Kuh & Ikenberry, 2009).Findings from the assessments are used as evidence when making decisions on curriculum review and classroom instruction to improve student progress (Banta, 2007;Bresciani, 2011).PLO assessment therefore helps improve student learning (University of California, Santa Cruz C, 2013) as it provides a structured way to document good practices (Kuh et al, 2007).
Performance Program learning outcomes are goals predetermined by instructors that represent the knowledge students should have when they complete their program (Banta et al., 1996).The assessment involves a systematic investigation of the programmatic goals, beyond the individual course level, to ascertain if and at what level the students are accomplishing those objectives.Program assessment gives students a better learning experience to develop competencies, knowledge, and skills; it enables faculty to evaluate the program successes and needs (Nyarigoti, 2020); and it guides departments on teaching effectiveness and future program directions.
Senior College and University Commission (WSCUC), specifies that undergraduate programs must ensure the development of five core competencies: Critical Thinking, Written communication, Oral Communication, Information Literacy and Quantitative skills.WSCUC further requires that institutions explain learning outcomes in relation to these core competencies and demonstrate the extent to which the outcomes are achieved.To meet this requirement, USIU-A through the General Education assessment conducts annual assessments on these competencies as a quality measure to ensure learning outcomes are achieved.Annual program assessments are intended to promote regular engagement in evidence-based program-level decision making and planning for student learning and success.
The General Education (GE) at USIU-A provides English courses at three levels: ENG 0999, ENG 1106, ENG 2206.At entry, students sit for a language placement test to assess their level of language competence.The scores are used to place students in their relevant classes where they can receive appropriate learning support.For English, students who do not meet the expected mark of 60% are rated as having a competence Below Expectation hence placed in a Basic Reading and Writing course (ENG 0999).This is a course that is aimed at supporting students with basic language skills that will help them start off in the courses they are enrolled.Those who pass the placement test get to enroll in Composition I (ENG 1106) class which is a level higher.
The ENG 1106 (Composition I) course covers academic writing, exposition, evaluating sources for their use of evidence, and citing specific textual evidence to support claims and explanations in their writing.Students are expected to attain a score of 70% to enroll for the final ENG 2206 course.ENG 2206 (Composition II) is the final level of the English courses.At this level, students are expected to comprehend and evaluate a variety of different texts and documents; construct effective arguments and explications of complex or multifaceted information; build and share their knowledge with others through writing; tailor communications to particular audiences, tasks, purposes, genres, and disciplines; select and use evidence that is appropriate for the discipline; conduct research and inquiry from multiple sources, evaluating their reliability and credibility; and using them appropriately to support claims in writing.
Previous assessments of written competency within the institution have always targeted students at their final year of their courses.These assessments measured the students' competency after their 4-year degree course, to find out whether the intended Program Learning Outcomes (PLOs) were achieved.Such an approach, however, tended to overlook various stages and learning processes such as the students' competence level at entry, and the progress made during the course.It did not allow any opportunity to identify and mitigate any challenges during the course before students graduate.To address this weakness, this assessment adopted a longitudinal approach.
This paper therefore presents analysis of a longitudinal assessment of students' competencies through the outlined course levels.The main objective was to Inform the institution about the aspects of writing that pose challenges for students, which could be addressed through instruction; measure student progress since registration attributable to the program or course of study; and provide evidence-based recommendations intended to improve the teaching and learning at USIU-Africa.It is important to note that the bulk of instruction of these courses occur in the first or second year with minimal writing instruction when students move on from general education courses to more specialized work within their major disciplines.It is however assumed that writing as a skill must be continuously integrated into curricula, so that students can learn to communicate effectively within the constraints and values of their discipline through exposure to and practice of the conventions and genres that are valued for success in that discipline.The assessment began at the English language entry test and ended at the third and last level of the English courses offered at the institution.Findings from this assessment were aimed at providing instructional relevant feedback to instructors which could be used to inform curricular and instructional improvement.

II. METHODOLOGY
Data for this assessment were obtained from students written essays.The essays were scored using a locally developed writing rubric.Kuh et al. (2015, pg. 39) notes that "rubrics encourage the use of authentic student work for assessment."The rubric that was adapted for the assessment was separated into four different domains of student writing: Relevance and content, Organization and development, effectiveness of the presentation of the essay, and the overall style of writing.Each of these dimensions was assigned a score of Below Expectation, Approaching Expectation, Meets Expectation and Exceeding expectation, these values representing increasing degrees of achievement in the criteria.All scripts were scored independently by three raters, with each rater scoring the artifact for each domain using a four-point scale.These individual domain scores were then averaged to provide an overall score for each student essay.
It was agreed that if 70% of the students attained the levels 'Meets Expectations' & 'Exceeds Expectations' in each of the standard, then the students had the requisite skill level.Similarly, if the aggregate score was 70% and above, then it could be assumed that the sample students had the requisite writing skills.The score could also indicate that the USIU-Africa General Education program was realizing written communication competency.The rubric used is presented in Table 1.
To measure student's competencies through the four levels, data was obtained from students' written tasks administered at the four levels i.e., from Placement Test to the final ENG 2206 course.Out of 308 students who sat for English placement test in Fall 2019, 119 scored 59% and below.These were placed in level 0999.The 119 students formed the target population for assessment with the main goal of monitoring their progress over three semesters of learning English.In the first phase of the assessment, 60 out of the 119 students placed in ENG 0999 in Fall Semester were randomly sampled to form the study population.The 60 were to be followed as they progressed through the four courses: Placement, ENG 0999, ENG 1106, andENG 2206.For this reason, the study lasted from Fall 2019 to Summer 2020.It was possible to conduct the first two phases in Fall 2019 because the study population was drawn from students who took both the placement test and ENG 0999 that semester.Therefore, placement test scripts samples were assessed at the beginning of the semester and ENG 0999 scripts were assessed at the end of the same semester.In the second phase (ENG 0999), a total of 56 student scripts were assessed.In phase 3 (ENG 1106), 43 scripts were assessed.Phase 4 of the study was significantly affected by the declined enrolment numbers in Summer 2020.Consequently, out of the 43 students involved in phase two of the study, only 36 enrolled for ENG 2206.The analysis in the next section is therefore based on the 36 students' sample who completed all the course levels from Placement, ENG 0999, ENG 1106 through to ENG 2206.The reduction of the students' numbers could be attributed to dropouts, course retake for those who did not attain the minimum requirement to move to the next level, or deferment.
At each phase, three assessors scored each written script independently.At the end, the team collectively reviewed the scores and in cases where there was a high variation in scores, they discussed and agreed on the most appropriate one.Although it is a general requirement to maintain the anonymity of participants in research, this was a longitudinal study, and the team was expected to keep track of the students as they progressed through the stages.Therefore, student's forms of identification such as names and school identification numbers were not erased from the scripts, but anonymity is maintained in this report.Using the rubric, the three team members scored the scripts using four competence levels: The writing engages the reader through an original prose style appropriate to the subject.Language is precise.Sentences are varied but not noticeably so.
The writing keeps the reader's attention through a carefully crafted prose style.Language chosen is appropriate to the subject but may call attention to itself in minor ways.
The writing is clear but could be expressed in a style more appropriate to the subject.It is jargon-free but may require a more complete explanation of some terms used.
The writing lacks clarity and is sometimes confusing.The language chosen is not appropriate to the subject or the assignment.

Students Writing Competency at Entry Level (Placement Test)
The results show that on the criteria of Relevance and content, 38 (22%) out of the 36 sampled students' scripts, were rated at below expectation, 20(56%) at approaching expectation, 8(22%) at Meets expectation and none at Exceeding expectation.A similar trend is seen in Organization and Development where 14(39%) of the students' essays were rated at Below expectation.18(50%) at Approaching Expectation, 4(11%) at meets expectation and none at exceeding expectation.The aggregate scores show that overall, 36% of the students were at below expectation, 47% at approaching, 17% at meets expectation and none at Exceeding Expectation.

Students' Writing Competence at ENG 0999 Level
The Assessment of students' written essays was conducted after 14 weeks of learning the course.All the students who were sampled from the placement test completed the ENG 0999 course.The results show a general improvement in all the four criteria.For instance, in the criteria of Relevance and content, 3(8%) were rated below expectation and 13(36%) Approaching Expectation, down from 38 (22%) below expectation, 20 (56%) Approaching Expectation and 14(39%) meets expectation and 6(17%) exceeding expectation up from 8(22%) and zero respectively at entry level.Below is a graphic representation of the scores.Figure 3

Students' Writing Competence at ENG 1106 Level
Unlike ENG 0999 where all sampled students from the placement enrolled for ENG 0999 level, in ENG 1106, the number dropped.This was attributed to several reasons: some students did not meet the required minimum score to proceed to the ENG 1106 level, others had deferred their studies due to Covid-19 related challenges, as others opted to take the course later.For these reasons, the sample number dropped from the initial 118 placement to 45 at ENG 1106 level and finally to 36 at ENG 2206 Level.Since it was a longitudinal assessment, only those who completed all the four levels were sampled for analysis.
The results presented here are for the students who met the minimum requirement of 70% and above in their ENG 0999 course.Assessment scores at level 1106 show general improvement in most of the criteria.In the criteria of Relevance and content, there were only 3(8.3%) of the students' scripts that were rated at Below expectation compared to 38 (22%) at entry and 8(22%) at 0999 level.10(27.8%)were rated at Approaching Expectation compared to 20(56%) at entry and 14(38.9%)at 0999 level.A total of 60.6% were at Meets and Exceeding expectation compared to 8(22%) at entry and 20 at 0999 level.A similar trend is seen in Organization and Development where 14(39%) of the students' essays were rated Below expectation, 18(50%) at Approaching Expectation, 4(11%) at meets expectation and none at exceeding expectation.Overall, 38% of the students were at below expectation and Approaching Expectation a reduction from 83% at entry and 61% at 0999 level.The total percentage for those rated at Meets and Exceeding Expectation rose to 58% up from 17% at entry and 39% at 0999 level.
The most challenging area in ENG 1106 level according to the rating was style where a combined total of 44.5% of the students' essays were scored at Below and Approaching Expectation.The students were doing best in Organization & development where a combined total of 63.9% were rated at Meets and Exceeding Expectation.This is however still below the expected score of 70%. Figure 4 below presents the scores per criteria.

Students' Writing Competence at ENG 2206 Level
The sample essays evaluated at this phase were for those students who met the 70% and above requirement in ENG 1106 to move to 2206 level.The assessment of the students, essays was conducted in Summer Semester of 2020.The results indicate that at this level, a total of 70% of the essays were rated at Meets and Exceeds Expectation, with the majority (51%) at Meets

Overall Competence on Organization and Development
The assessment criteria described an essay that meets the required expectation to have sound logic.It should be a unified essay that proceeds coherently with an effective introduction, well-developed and unified paragraphs, effective transitions, and a conclusion that, rather than summarizing previous points, explores the implications of the preceding analysis.Results indicate that overall, in Organization and Development, students attained the expected learning outcome at level 2206 having a total rating of 72.3% at Meets and Exceeds Expectation.At entry level, a total of 89% were at Approaching or Below Expectation and 11% at Meets expectation in this criterion.At level 0999, 61% were rated at Approaching and Below Expectation while 39% were at Meets and Exceeds Expectation.At Level 1106, a total of 32.2% were rated at Approaching and Below Expectation and 63.9% at Meets and Exceeds Expectation.These results show consistent improvement from entry though the students are just able to attain required competence in the criteria at level 2206.The results are presented in the Figure 7 below.

Overall Scores: Syntax and Grammar
A student who has reached the expected competence level in Syntax and Grammar uses language skillfully to communicate meaning to readers with clarity and fluency, and the writing is virtually error-free.The results indicate that overall, students seem to do poorly in this domain.At level 2206, the students did not seem to have attained the expected competence level.Only 62% of the students' essays were rated at Meets and Exceeding Expectation which was below the required 70% mark set by the assessment team.At entry level, a total of 83% were at Approaching or Below Expectation and 17% at Meets expectation in this criterion.Worth to note is that this score is an improvement from 67% Approaching and Below Expectation and 34% Meets and Exceeds Expectation at level 0999.It is also an improvement from a combined 41.5% of Approaching and Below Expectation and 55.5% of Meets and Exceeds Expectation at Level 1106.The improvement, however, is not good enough to get the students to the expected level (above 70%).Figure 8  As per the criteria used in this assessment, an essay that meets the expected competence standard in style engages the reader through an original prose style appropriate to the subject.The writer uses varied sentences and adopts precise language appropriate to the subject.Results show slow progress in Style.Although there was no student rated at Below Expectation, a high percentage (36.1%)were at Approaching Expectation, bringing the combined competence of Meets and Exceeding Expectation below the required 70%.This therefore implies that at 2206 level, the students had not attained the expected learning outcome in the criteria of Style.
At entry level, a total of 83% were at Approaching or Below Expectation and 17% at Meets expectation in the criteria of Style.At level 0999, 67% were at Approaching and Below Expectation while 34% were at Meets and Exceeds Expectation.At Level 1106, a total of 41.5% were at Approaching and Below Expectation and 55.5% at Meets and Exceeds Expectation.See Figure 9

Students' Overall Proficiency in Writing
Results for the students' overall performance for all course levels show that at placement, there were 36% of the students rated at Below Expectation, 47% at Approaching Expectation, 17% at meets Expectation and none at Exceeding Expectation.At level 0999, there were 20% of students at Below Expectation, 41% at Approaching Expectation, 30% at meets Expectation and 9% at Exceeding Expectation.At level 1106, there were 8% of students at Below Expectation, 31% at Approaching Expectation, 51% at meets Expectation and 8% at Exceeding Expectation.At level 2206, there was 1% of students at Below Expectation, 29% at Approaching Expectation, 51% at meets Expectation and 19% at Exceeding Expectation.The results are presented in Figure 10 below.

Figure 10: Percentage of Students' Proficiency in Writing
To evaluate whether the students are attaining the expected learning outcome in written communication, the assessment team used the final 2206 level scores.These results may not however be comprehensive and conclusive.This is because most of the students complete their English courses in their second year and it is expected that they will continue to gain more writing skills in the third and fourth years of their studies as they write varied tasks in their major courses.
Although the results at level 2206 (70%) appear to be acceptable overall, Grammar and syntax is one area of weakness requiring further action at the institutional level.The students did not perform up to expectations in the criteria (below 70%).

IV. DISCUSSION
This study emphasizes the significance of longitudinal PLO assessment as opposed to course summative assessments.Longitudinal assessment takes place cyclically over an extended period, often years.A cyclical process allows for creativity, innovation, and accountability within a program (Bresciani, 2011).As noted earlier in this paper, previous assessments of written competency within USIU-A targeted students' achievements at their final year of their courses.This approach disregarded various stages and learning processes such as the students' competence level at entry, and the progress made during the course.Students benefit more from longitudinal assessment as opposed to the end of course summative assessment as it provides the opportunity to identify and mitigate any gaps during the course before students graduate.The process often requires regular review of assessment tools and practices, and modification of programs, based on previous assessment results.This PLO assessment in this study was criterion based, implying the evidence collected was analyzed based on specific competencies and skills.This type of assessment is different from student evaluation.In student evaluation, the instructor uses methods such as quizzes, assignments, exams, worksheets, clinical evaluations, group or individual projects, etc., to judge student learning and understanding of the content for purposes of grading and reporting.Students receive feedback from the instructor about their learning.PLO assessment on the other hand uses analytical rubrics that breaks down a specific PLO into essential dimensions or traits to observe and measure strengths and weaknesses across various criteria (Oakleaf, 2009).Rubrics, also known as scoring guides or grading criteria, are useful tools for clarifying learning outcomes and communicating performance expectations A rubric constitutes a description of a task, evaluation criteria, the levels of the task that must be met to succeed, and a scale of evaluation (Council of Graduate Schools, 2011).Criterion rubric seeks to find out whether the student meets the expected standard of performance rather than, how well the student does, compared to other students.In this study the rubric competence descriptors (Relevance and content, Organization & development, Syntax and grammar, and Style) were scored as Exceeds expectation, Meet expectation, Approaching Expectation and Below Expectation.
The results show that, in general, the students did improve their writing over their 4 English course levels.A combined percentage of meets and Exceeding Expectation at this level is 70% implying the student are just at the pass mark in achieving the writing communication competence.This means there is still room for improvement in this area.The results also indicate that most students are only able to attain the expected outcomes at level 2206 suggesting that students improve their writing skills as they progress.This is in line with a study by Hilgers and Kissling (2016) that found that students in their final year of undergraduate studies demonstrated significant improvement in their academic writing skills compared to their first year.The authors attributed this improvement to the students' exposure to different writing genres and the development of their critical thinking skills.
The students demonstrated significant improvement in the area of 'Relevance and content' with 83.4% of students achieving a Meets and Exceeds Expectation rating.At the entry level, only 22% of students were at this level, but this rose to 70.9% at Level 1106 and continued to improve at higher levels.This finding is supported by existing literature, which suggests that many students entering higher education struggle with writing skills.For example, recent studies have found that first-year college students still exhibit significant deficiencies in their writing skills (Brauer, Ferguson, & Ball, 2018;Liu & Gao, 2019).These studies have highlighted issues with sentence structure, grammar, punctuation, and coherence.Similarly, research conducted by Brown and Hood (2021) has found that students entering higher education still struggle with writing clearly and concisely, and their writing often lacks coherence and organization.
The study also showed that the area where students struggle the most is Syntax and Grammar, with only 62% of students achieving a Meets and Exceeds Expectation rating at the highest level of study.This is below the required 70% mark set by the assessment team.However, there was an improvement from 67% Approaching and Below Expectation at level 0999, indicating that students are making some progress in this area.In terms of Style, the results show slow progress, with 36.1% of students at Approaching Expectation and none at Below Expectation.However, this is not enough to achieve the required 70% Meets and Exceeds Expectation rating.

V. CONCLUSION
Results of the assessment showed that 70%f students' papers reflect 'Meets or Exceeds Expectation' writing abilities.The consistence low scores in some domains such as syntax and grammar in all course levels suggests that writing across the curriculum could have a substantial impact on students' writing competency, especially after their last English course.After having considerable writing skills in their English courses, student writers are expected to produce different text structures, for different audiences, with various goals or purposes.However, development of these skills is dependent on students' experiences with various instructional strategies.Results from this assessment therefore are not conclusive, because after their English courses, the students still have more semesters specializing in their majors.During this period, their writing may improve or drop.Another assessment at their final year would give a more reliable result of whether USIU-A is achieving the expected learning outcome in written communication.This assessment has however succeeded in giving insights to the status and challenges faced at the distinct levels of English courses since entry.This assessment made it is possible to propose strategies to mitigate any challenges experienced at each level to ensure students achieve the expected learning outcomes.

VI. RECOMMENDATIONS
The purpose of this study was to evaluate whether students at USIU-Africa were achieving the intended learning outcomes of the English course and make recommendations for improvement on teaching methods or course materials.Therefore, based on the findings of the assessment, the following recommendations were made: Firstly, a language Intensive program before students begin their degree program would address the issues of course retake or non-completion.Students who are very weak in English may struggle to understand and communicate effectively, leading to lower grades and increased risk of course retake or non-completion.To address this issue, the university can provide more language support for these students through a language intensive program before the start of their degree program.Such a program would focus on developing students' reading, writing, listening, and speaking skills in English.By providing this additional language support, the university can help ensure that students have the necessary language skills to succeed in their degree program.
Secondly, the institution should promote writing across the curriculum (WAC).This is an approach that promotes writing as a tool for learning and critical thinking in all academic disciplines.Through WAC, students can learn to communicate effectively within the constraints of their discipline by being exposed to and practicing the conventions and genres in that discipline.This approach can help students to develop their writing skills in a way that is relevant and meaningful to their academic and professional pursuits.
Thirdly, the institution needs to limit class size in language courses to 30.In language courses, it is essential to have a manageable class size to ensure effective teaching, evaluation, and feedback.A class size of 30 or fewer allows for more individualized attention and engagement with each student.Smaller class sizes also provide opportunities for more student participation and interaction, leading to a more dynamic and engaging learning environment.Additionally, limiting class size can help instructors provide more detailed and personalized feedback to students on their writing and language skills, ultimately leading to better learning outcomes.for effective teaching, evaluation, and feedback.
Finally, the institution could consider developing a program in English for specific purposes (ESP).This is a specialized approach to language teaching that focuses on the language and communication needs of learners in a specific field or discipline.By developing a program in English for specific purposes, universities can address the unique language and communication needs of students in different fields.This approach can help students to develop the language skills and knowledge necessary to succeed in their chosen field, whether it be business, engineering, or medicine.Moreover, an ESP program can help students to communicate effectively with colleagues and clients in their field, leading to greater success and impact in their careers.to address discipline specific needs.

:
Results of Students Writing Competence at ENG 0999 The ratings show improvement in all the domains assessed.Only 1% were at Below Expectation, 29% at Approaching Expectation.Syntax and grammar caused the greatest challenge as 14(38.8%) of the students' essays were rated at Approaching Expectation, followed closely by Style at 36.1%.There was no student rated at Below Expectation in the criteria of relevance & content, Syntax & Grammar, and Style and only one student in organization & development.The students were doing well at Relevance and content where 83.4% were rated at Meets and Exceeds expectation.The results are summarized in Figure5below.Students who have reached optimum level of competence in Relevance and content exhibit a keen sense of purpose and display a clearly defined focus in their written work.The work exhibits depth and complexity in its analysis and originality of thought.It offers sufficient and appropriate support for the thesis in the form of concrete, specific and relevant evidence.Overall, in Relevance and content, students seemed to attain the required competence having a total rating of 83.4% of a combined Meets and Exceeds Expectation.Improvement is seen from placement to 2206 level.At entry level, a total of 78% were at Approaching or Below Expectation and 22% at Meets expectation in this criterion.At level 0999, 44.3% were at Approaching and Below while 56% were at Meets and Exceeds Expectation.At Level 1106, a total of 36.1% were at Approaching and Below Expectation and 70.9% at Meets and Exceeds Expectation.Below is a graphic representation of the ratings.
below give a graphical representation of the results of the four levels. below.