“Learning another language is not only learning different words for the same things, but learning another way to think about things.” ‒ Flora Lewis

ASSESSMENT DESIGN RATIONALE

​

Background Information

 

This assessment is designed to test students who are enrolling in a 44-week Vietnamese language course. Students take this test after having completed an eight week intensive language training period which requires five hours of classroom instruction and three hours of self-study per day. This test is designed to test Reading, Listening, Vocabulary, and Speaking Skills. The test result will be used to identify student’s strengths and weaknesses and for teachers to give them feedback on how they have done and what they need to focus on in the upcoming weeks. Based on the test results, administrators might regroup classes so students will be placed in groups of similar language ability.

​

This test is considered as both formative and summative assessment. The results will give all stakeholders an idea of strengths and weaknesses of students. It will help stakeholders to know how much and how well students have acquired the language. When necessary, teacher and students can work together to modify the curriculum for students in the future. Modification of curriculum might be to slow down if students are struggling or speed up if the pace is too slow.

​

At my school, classroom sizes are relatively small, varying from two to four students per class. Instructors work very closely with students and with other instructors in order to identify strengths and weaknesses of each student very early in the program. Our students are working professionals with specific learning goals that include the ability to communicate in Vietnamese language with native speakers in various contexts, such as meeting with business partners, local employees, going out shopping, dining, and interviewing local people.  The second goal of our students is to pass the proficiency tests in speaking and reading at the end of their training.

​

Overall Design

 

This test consists of four parts: Reading, Listening, Vocabulary and Speaking. Each part aims to test its language construct through tasks that show students' language ability. In the reading and listening test parts, I create test prompts and ask students to respond in English in order to avoid possible misunderstanding of test prompts and to prevent students copying from the reading text.

During the first eight week of training, the program covers various day-to-day topics including self-introduction, biographical information, occupation, contact information, family, time, likes and dislikes, foreign language, daily routines, and travel experience. As a result this achievement test is designed to measure what students have achieved during the first eight weeks of study.

​

For Reading skills, students are required to read a passage in Vietnamese and answer four comprehension questions and complete four true/false statements in English. The language of the text is appropriate to students' estimated proficiency level and reflects the language and topics that students have read in the program up to the time of this assessment.

For Listening skills, students will listen to an audio recording twice and answer four comprehension questions in English. Since students are at the beginning level, I feel it is better to allow students to listen to the recording twice. This will give them a second chance to catch the information they need and double check the information they think they have heard.

To assess Vocabulary skills, I use a picture as a prompt for vocabulary that students are expected to fill in the blanks with. Pictures show daily activities of a person and writing prompts require students to use specific words indicating action and time as shown in the pictures.

To assess speaking skills, I create a scenario where students will meet a native speaker who comes from out of town. Students have to engage in a conversation and exchange information listed in the test prompt including exchanging greetings, nationality, phone number and address, occupation, foreign language(s) they can speak, likes/dislikes and daily routines, travel experience, and other information for a bonus point. 

 

Administration

 

Test format: All students will take the Reading, Listening, and Vocabulary test parts at the same time in the classroom setting managed by one proctor. The speaking part will be conducted in face-to-face format. For the first three parts, paper test materials will be handed out to students at the beginning of the test. Audio recording for the listening test part is uploaded to the program intranet site prior to the test date. The speaking part requires two teachers. One gives instruction, keeps track of time, audio-records, and takes note of the performance. The second teacher participates in the role-play with the student and the two teachers co-evaluate the student’s performance at the end of the performance.

 

Duration of test: The test performance will take approximately 50 minutes to complete. Reading: 15 minutes, Listening: 10 minutes, Vocabulary: 10 minutes, and Speaking: 15 minutes. The test evaluation will be done by the same two teachers and expected to be complete in about 20 minutes.

 

Scoring: Key and rubric for scoring is attached for teachers as guidelines. The test results will be concluded right after the evaluation is complete and will be sent to the program supervisor in a timely manner. The test performance will be audio-taped for record-keeping purposes. The test results will be the composite of four test parts and ranked in three levels: High Pass (36-40 points), Pass ( 25-35 points), and Need Attention (<25 points). (See test key and rubric for more information).

 

Rationale

 

Content validity

According to Hughes (2003. p.26) “a test is said to have content validity if its content constitutes a representative sample of the language skills, structures, etc. with which it is meant to be concerned”. In my teaching context, we focus on Reading, Listening, Vocabulary, and Speaking skills. All instructional materials are designed and used to improve students' skills in these areas. I design tasks to measure the language that has been introduced in the curriculum, and content of the test reflects the materials taught during the course. In other words, this test is designed to assess how much of the course materials students have acquired during the first eight weeks. Testing the language students have learned, according to Hughes (2003), is a way to increase the test content validity.

 

Reliability

As defined by Hughes (2003), “all tests have a standard error of measurement which means that a test result is only an estimate of learner’s 'true score' and that learner’s 'true score' is unknown.” Measurement error refers to factors that impact the test results such as student’s background knowledge, rater’s bias, lucky guess, etc. Those factors could lead to the possibility of students getting different scores on the same test if they take it on a different day. Reliability of a test can be quantified in the form of correlation between test results called reliability coefficient (Hughes, 2003. p.41) which ranges from zero to one. In this range, zero reliability coefficient means the results of the two tests are completely different and one means the results of the test are the same regardless of when the test is taken. Reliability can be increased by including test tasks that cover a variety of language constructs, and eliciting samples of language skills being tested.

With this knowledge in mind, I created tasks that reflect the language constructs that I plan to test. For example, in my reading assessment, students need to respond to the test items in order to show their reading comprehension. Similarly, they have to show their listening ability by answering comprehension questions after listening to a passage during the test. For Vocabulary test, the purpose is to see whether students remember and are able to use the correct vocabulary in the right context. I restrict their possible choices of vocabulary by using a graphic prompt which means students have to use the vocabulary that corresponds to the picture to answer the prompts. Lastly, students are required to engage in a conversation in order to show their oral communicative proficiency. Although the reliability coefficient value would not be one, the maximum value, this test will give stakeholders a very good idea of how much and how well students have learned during the first eight weeks. It is important to note that in the Reading and Listening test parts, students are allowed to use English to respond to the test prompts which ensure their understanding is not limited by their target language proficiency.

 

Testing techniques

I employ several common testing techniques (Hughes, 2003) in this test including Wh-questions, True/False items, gap filling, and role-play in different test parts.

 

In the reading part, I create Wh-questions with a goal to elicit information that students need to draw from their comprehension of the reading passage. True/False items also require students to understand the information in the test items and to be able to locate it in the reading passage. I paraphrase the information which makes it more challenging for them. One drawback of True/False items is that “there is a 50% chance of choosing the correct response” (Hughes, 2003, p.79). However, this is a low-stake test and therefore it is not necessary to eliminate this type of test item. In addition, True/False only accounts for a part of the reading result as students need to complete the Wh-question portion of the test as well.

 

In the listening part, students will listen to the audio recording twice and answer four comprehension questions. Since students are in the beginning phase of training, I think it is very easy for a beginner to miss information after the first listening. Giving students the second chance will boost the possibility of getting better responses to test items which leads to bringing more positive impact on their study.

In the writing portion, students will respond to test items in this part by completing six gap filling items. Since this type of test item is likely to have more than one correct option, I also use a graphic prompt to restrict the option students may choose. For example, at a certain time the person perform an action that is shown in the pictures.

 

In the speaking part, I use the role-play method in order to measure students' speaking ability. According to Phan (2008) “communicative language tests are used with the goal of measuring language the learner's ability to take part in acts of communication or to use language in real life situations” (p.4). I adopt this communicative language testing approach and incorporate it into the scenario for students to role-play. In doing this type of task, sometimes students forget that they should be interacting as if they were in a real conversation because they are consciously aware that they need to cover all items listed in the prompt. However, the setting for this test is semi-authentic which reflects the real life situations that our students will encounter in the future. It should give teachers a good grasp of students' language ability.

 

One aspect of assessing speaking ability that I would also like to focus on is pragmatic competence. According to Roever (2011), Crystal (1997) defined pragmatics as “the study of language from the point of view of users, especially of the choices they make, the constraints they encounter in using language in social interaction and the effects their use of language have on other participants in the act of communication” (p.301). In this Speaking test part, I do not add “pragmatics” as one of the separate criteria to evaluate speaking ability but rather it is assessed under item C (Vocabulary) (See key and rubric for more information), which focuses on the appropriateness of words and phrases students use in this specific context. Since students have completed the first of five phases of their study, in my opinion, it is appropriate to start assessing pragmatic competence in their interaction with native speakers. It shows students' ability to engage and the level of success in conveying their messages.        

 

Conclusion       

In conclusion, I design this test to assess students’ language ability in all four skills including Reading, Listening, Vocabulary, and Speaking. I create tasks that cover the language students have studied in the course in order to increase the test validity. I make sure that teachers will be able to elicit ratable samples of the language skills that are being tested in each part. The results of this assessment will give all stakeholders a good indication of each student's  language ability and will be used to make necessary adjustments to the program. 

 

References

Cheng, L. & Curtis, A. (2012). The Cambridge Guide to Second Language Assessment (p.89-95).

Cambridge, UK: Cambridge University Press.

Crystal, D. (1997). A dictionary of linguistics and phonetics. Oxford, UK: Basil Blackwell.

Douglas, D. (2014). Understanding Language Testing (p1-4). New York, USA: Routledge.

Hughes, A. (2003). Testing for Language Teacher. Cambridge, UK: Cambridge University Press.

Phan, S. (n.d.). Communicative Language Teaching. Retrieved August 8, 2016, from            https://www.hpu.edu/CHSS/English/TESOL/ProfessionalDevelopment/200820TWPsprin g08/6_1_02Phan.pdf

Roever, C. (2011). Testing of second language pragmatics: Past and Future. Language Testing,

28(4), 463-481. Doi: 10.1177/0265532210394633

​

 

Appendix A: Test Materials

Eight – week Progress Evaluation

 

Part 1: Reading (10 points - 15 minutes):

Read the text below and answer the following question:

​

Anh Quang là hướng dẫn viên cho công ty du lịch Hải Hà. Anh ấy sống ở trung tâm thành phố Hà Nội cùng với vợ và hai con gái. Anh ấy sống ở một căn nhà nhỏ ở quận Hoàn Kiếm, gần Hồ Gươm. Hàng ngày anh ấy đi làm việc bằng xe máy lúc 6 giờ sáng và về nhà lúc 5 giờ chiều. Vợ anh ấy thường ở nhà, nấu ăn, chăm sóc hai đứa con và dọn dẹp nhà cửa. Anh Quang ít khi về nhà ăn trưa vì chỉ có 45 phút nghỉ trưa. Anh ấy thường ăn cùng với đồng nghiệp ở những quán ăn nhỏ gần cơ quan. Anh ấy luôn cảm thấy hạnh phúc và tự hào về gia đình của mình.

Chị Hương và chồng của chị ấy, anh Hùng sống tại một ngôi nhà vừa to vừa đẹp ở ngoai ô thành phố. Họ mới lập gia đình 5 tháng trước và chưa có con. Chị Hương là phóng viên của báo Phụ nữ Hà Nội và chồng chị ấy là Trợ lý Giám đốc ở khách sạn Quê Hương.

Thỉnh thoảng vào các ngày nghỉ lễ gia đình anh Quang và gia đình chị Hương cùng nhau đi du lịch ở các thành phố trong nước như Nha Trang, Huế. Tháng trước hai gia đình đã đi thăm Hàn Quốc trong 2 tuần. Họ đã đi tham quan nhiều thắng cảnh nổi tiếng và ăn các món ăn truyền thống của người Hàn Quốc. 

 

Answer Questions

 

1. How many people are there in Quang’s family?

​

2. What does Quang’s wife usually do?

 

3. Where does Hương work?

 

4. Where did they go last month?

 

True or False?

​

1. Quang usually goes home for lunch.

2. Hương and Hùng have two sons.

3. Hùng works as an assistant at Quê Hương hotel.

4. They sometimes travel abroad together.

 

 

Part 2: Listening (10 points - 10 minutes)

Listen to this audio recording TWICE and answer the following questions:

 

1. What does Nam do for living?

 

 

1. Does he like his job? Why/why not?

 

 

1. What time does he finish work?

 

 

1. What does he usually do in the evening?

 

 

Part 3: Vocabulary (10 points- 10 minutes)

Look at the pictures and complete sentences below

​

1. Anh ấy ………………………………… lúc 6 giờ sáng.

2. Anh ấy ………………………………… lúc 6 giờ 30.

3. Anh ấy …………………………………. lúc 7 giờ.

4. …………………………… đi làm lúc ………………

5. Anh ấy ………………………………….lúc 7 giờ 30.

6. Anh ấy ………………………….. tối.

 

 

Part 4: Speaking (10 points - 15 minutes)

​

You are meeting a Vietnamese guest at a friend’s party in the Washington D.C area. The guest is visiting the U.S for the first time. Please have a conversation with the person and exchange the information listed below.

​

(Teacher plays the role of the guest)

• Greetings, nationality

• Phone number, address

• Work experience: position, where, how long

• Foreign language ability

• Like, dislike, daily routines

• Travel experience

• Other information

 

• Note to students: you will be evaluated basing on the following factors

• Pronunciation

• Grammatical structure

• Vocabulary use

• Fluency

 

​

Appendix B: Listening Script (For Teacher only)

 

Anh Nam là phóng viên báo Người Hà Nội. Buổi sáng, anh ấy thường thức dậy rất sớm. Sau đó anh ấy ăn sáng và đến cơ quan làm việc lúc 7 giờ kém 15. Công việc của anh ấy rất thú vị vì anh ấy có cơ hội làm việc với rất nhiều người. Anh ấy thường viết về cuộc sống, xã hội và lịch sử Việt Nam. Anh ấy thỉnh thoảng đi ăn trưa với bạn đồng nghiệp. Buổi chiều, anh ấy thường làm việc từ 1 rưỡi đến 4 giờ. Buổi tối, anh ấy thường đọc sách để tìm hiểu về văn hoá và lịch sử thế giới. Vào các ngày cuổi tuần, anh ấy thường đi nhà thờ cùng với gia đình, đi thăm các bảo tàng hay trung tâm mua sắm trong thành phố.

 

 

Appendix C: Assessment Key and Speaking Rubric

​

Part 1: Reading (15 minutes) (10 points)

Answer the following questions (6points)

1. Four people (1)

2. Stays home, cooks, takes care of kids, and cleans the house. (3)

3. Hanoi Women’s newspaper (1)

4. Korea. (1)

   1. True or False (4 points)

5. F (1)

6. F (1)

7. T (1)

8. T (1)

Part 2: Listening (10 minutes) (10 points)

Listen and answer the following questions

1. Reporter/Journalist (2)

2. Yes. Because it’s interesting and he has an opportunity to meet people (4)

3. 4:00pm (2)

4. He reads books (2)

 

Part 3: Vocabulary (10 minutes) (10 points) – Look at the pictures and complete sentences below

1. Thức dậy (1)

2. Tập thể dục (1)

3. Ăn sáng, uống cà phê (2)

4. Anh ấy, 7 giờ 30 (3)

5. Ăn tối. (1)

6. Lúc 8 giờ 30. (2)

 

Part 4: Speaking (15 minutes) (10 points)

Performance Factors/Items covered

Pronunciation (A)

Grammar (B)

Vocabulary (C)

Fluency (D)

​

Comments
(reasons for rating)

1. Greeting & nationality: 0.50

2. Phone number & address : 1.00

3. Work: position, where, when, how long on the job: 1.50

4. Foreign language ability: 2.00

5. Likes, dislikes, daily routines: 2.00

6. Travel experience: 2.00 

7. Bonus: Other information: 1.00

​

Total points: 10.00

​

Note to teachers

• Students earn points if he/she can get his/her message across and show the ability to follow the conversation.

• Point in each evaluation factors ranges from 1-3. 3 means High Pass, 2 means Pass, and 1 means Need Attention

• (A) Pronunciation: 3 accurate, 2 comprehensible, and 1 incomprehensible

• (B) Use Grammar structure correctly: 3 (>80%), 2 (40%-79%), 1 (<39%)

• (C) Choice of vocabulary: 3 appropriate, 2 acceptable, and 1 unacceptable

• (D) This factor is for feedback, not scoring because students are in the first phase, most are unable to speak at ease yet.

 

 

Total score:

8.60-10.00 points: High Pass

7.00-8.50 points: Pass

<7.00 points: Need Attention

 

​

Appendix D: Sample of Completed Speaking Rubric

 

Performance Factors/Items covered

Pronunciation (A)

Grammar (B)

Vocabulary (C)

Fluency (D)

​

Comments
(reasons for rating)

1. Greeting & nationality: 0.50

2. Phone number & address : 1.00

3. Work: position, where, when, how long on the job: 1.00

4. Foreign language ability: 1.50

5. Likes, dislikes, daily routines: 1.50

6. Travel experience: 1.50 

7. Bonus: Other information: 1.00

​

Total points: 8.00 / Pass

​