Cycle 1 Data

AR Focus Statement

Within the English classroom at Crenshaw School, students struggle to create relevant discourse regarding author’s purpose in assigned readings. Implementing the Reflect, Inquire, Suggest, Elevate (RISE) model of critical feedback as a requirement for students to follow when evaluating peer responses to discussion posts should enhance student analysis of an author’s purpose in assigned readings.

Inquiry Questions

Will my sixth and seventh grade target students be able to create explanations that reflect a deeper understanding of an author’s universal message when providing peer critiques?

Will my sixth and seventh grade target students’ responses during discussions become elevated when discussions are conducted independent of the teacher?

Will my 6th and 7th grade target students utilization of the RISE model of discourse increase the level of questioning among themselves during peer critiques and evaluations?

Target Audience

The target audience addressed in the Capstone project is 25 sixth, seventh, and eighth grade students at Crenshaw School in Galveston Independent School District. Sixteen are female students and nine are male students. Sixteen students are 12 years old. Nine students are 13 years old. Ninety-two percent of students are considered Low Socio Economic, and forty percent of students reside in households where Spanish is the primary language spoken. The students are in a combined Pre-Advanced Placement class. The environment of implementation was my classroom during scheduled class time. Cycle 1 consists of a group of 12 sixth and seventh graders.  Cycle 2 consists of a group of 13 seventh and eighth graders.

 Summary of Cycle 1

Students completed a pre-assessment that required students to analyze the level of responses in class projects and student critiques prior to RISE implementation.  Students evaluated responses to determine if the responses where literal, contained inference, or connected to a universal message.  Students also evaluated peer critiques to determine if students independently reflected a connection to peer projects and ideas, inquired for more information, suggested alternative ideas and different perspectives, and elevated connections to a new perspective or a societal meaning.

After completion of this survey, participating students viewed a video explaining the RISE process, analyzed sample RISE responses, and then deconstructed their own previous responses in former projects and discussion posts.  Students deconstructed these responses in a Prezi.  When students did not find connecting information that established the critical feedback process of Reflect, Inquire, Suggest, and Elevate, students created these types of responses in their Deconstructing RISE Prezis.  Upon completion of these projects, I generated questions to create a dialogue about elements of a new novel, Anthem by Ayn Rand. These new questions were posted in our course located in Schoology. The next week, students then responded to these questions on the discussion forum, knowing that their peers would provide critical feedback utilizing the RISE responses.  Week three students began implementing crucial feedback to two peer responses following the RISE format in Schoology.  In addition, students then generated their own upper level questions, and posted these questions in a new discussion forum.  These questions were limited to reflect upper level Bloom’s questions.  All questions were required to analyze, evaluate, or synthesize information within Anthem.

The final week, students began responding to peer questions.  Each question posed by peers had to be answered by all students participating in Cycle 1.  After completion of responses, students then implemented the RISE feedback process to two more student responses.  Upon completion of the RISE responses, students then completed the Post Assessment.  Students reflected back to his/her responses in their first Assessment and their previous discussion posts that did not require student implementation of the RISE feedback process.  Students then evaluated whether the new responses remained in the literal level of thought, inference level, or universal area.  Participating students then evaluated the implementation of RISE feedback to peer posts.  Each student evaluated whether he/she included a reflecting statement, at least one inquiry question, provided a suggestion for peers to consider, and elevated a new understanding or societal message for the peer response he/she responded to.

Data Collection

The pre and post assessment tools are surveys designed to have students independently assess their own level of responses.  Students determined whether responses were literal, inferential, and/or universal.  Both assessments also required students to independently analyze the level of critical feedback that he/she provided to peers.  In addition, I evaluated peer responses utilizing two rubrics.  One rubric evaluated what level of responses (literal, inference, universal) students provided when answering discussion post questions. The second rubric determined what level of feedback did each student implement during critical feedback to peer responses.  Both rubrics were utilized to evaluate discussion post responses that occurred prior to implementation of the RISE feedback process for We the Living by Ayn Rand.  These rubrics were also utilized to gather data for responses and peer feedback after implementation of the RISE feedback process for discussions associated with a new class novel, Anthem by Ayn Rand.  Students participated in three trials.  The first trial was pre RISE.  The second trial occurred a week after implementation of the RISE feedback process. The third and final trial was implemented two and a half weeks after implementing the RISE critical feedback process.

Data Report

Students indicated in the pre-assessment that all students felt responses and projects primarily demonstrated inference.  No students indicated that they considered their pre RISE responses and projects to contain universal connections.  One finding that is evident based on question one of both assessments is that what students initially deemed as insightful responses indicating inferential connections was faulty. Question one of the Pre-Assessment indicated more utilization of inference while the Post-Assessment responses indicated that the projects and posts prior to the implementation of RISE were only somewhat insightful.

When comparing the data in the pre-assessment with the post assessment in regards to providing critical feedback to peers, the pre-assessment indicated that over 50% of students felt that their critiques elevated peer responses. However, on the post-assessment that required students to again evaluate critical feedback prior to the utilization of RISE, students indicated that the majority of feedback was limited to providing primarily reflective statements.  Given this information, student interpretation of what reflect, inquire, suggest, and elevate prior to its implementation was faulty.

The data in regards to RISE implementation, student responses indicated that 50% more students provided reflective statements in peer feedback. While comparing the data from the survey, there is one common anomaly: student evaluations of response levels changed after implementation of the RISE feedback process.  This shift in responses, leads me to determine that the assessments are not conducive to providing valid responses when comparing them.  However, the responses in the post survey do demonstrate some valid information.  Student critiques of responses in this assessment showed a significant increase of analysis and feedback.  Whereas students indicated that the majority of responses, prior to RISE implementation, stayed in the somewhat insightful/slightly inferential area;  however, after implementation, the final discussion post responses students determined to fall under the category of mostly insightful.  In the initial evaluation on this assessment, students indicated that peer feedback was limited to reflective comments only; however, after three weeks of implementation of the RISE feedback process, students determined that the majority of feedback reflected, inquired, and suggested more from peers.  After three weeks of implementation, only twenty-five percent of students indicated that the critical feedback provided by students elevated a peer’s understanding.

Although the comparative analysis from the Pre and Post Assessments generated faulty and unreliable comparisons, the Post-Assessment alone generated valid data.  Another tool I used to collect data was rubrics.  The level of responses rubric and the RISE rubric demonstrated a more reliable comparison.  After each student response to questions, I evaluated the responses based on literal (1), inference (2), and universal levels (3).  Each level a student reached earned that corresponding score. The same process was utilized for Reflect (1), Inquire (2), Suggest (3), and Elevate (4).  This information was recorded in a spreadsheet.  Trial 1 was prior to RISE implementation, Trial 2 was the week after RISE implementation, and Trial 3 was two weeks after RISE implementation.

Insight

Based on the data, Cycle 1 was relevant to my focus statement because it did generate increased connections and more insightful responses in relationship to text analysis.  Student responses were more insightful and the discussion posts engaged students through the implementation of RISE.

Based on the data analysis provided above, all of the inquire questions were addressed.  Students did demonstrate a deeper understanding of an author’s message, student responses were somewhat elevated (more than what they had previously been prior to the utilization of RISE), and student questioning definitely increased.  When analyzing the kinds of questions, the majority were upper level Blooms questions.

For example, the following is a typical student response prior the implementation of RISE:

“If the people are rude, greedy and wishing to destroy you then go with what you think but if they are nice then go with them.” (AC)

“Going the wrong way would cause a bad end.” (LT)

“The main character struggles with getting along with people. He likes to argue with other people around him. He does not like following the rules either.” (JH)

The following are examples of student responses after RISE implementation:

“That is not exactly their names, but it is kind of close. It is more like a brand. Have you ever seen a cow with a mark on him? It is like the same thing. When you brand cows, they have a group marking of some kind, then they have the identification mark. Names are a way of identifying a person through a unique ‘label’. Since it is a crime to be different, they cannot have names. Instead the ‘brand’ shows the group of brothers and then the identifying mark. For example, ‘Equality’ would be the group, and 7-2521 would be the identifying mark.” (JH)

“The numbers are like a tag. They also show the current population of their city. It serves as no thing of a name, because it states that giving someone a name is illegal. The other idea is that it could be month and year they were born. Ego, by definition, is, “the “I” or self of any person; a person as thinking, feeling, and willing, and distinguishing itself from the selves of others and from objects of its thought. ” Though, I take it as thinking highly of oneself and being able to admit your own individual actions.” (LT)

“It has already been stated in this book that all men live for the better of men. They cannot be different from their brothers, for it is a sin and the root of evil- that is what the Council of Vocations believe. So, whenever Prometheus stepped out of line to present his own, individual work, the council thought he was evil. They thought he was evil because it had been drilled into their brain that individuality is evil. So when they saw that Prometheus made something on his own, alone- they saw him as the devil. In the councils, messed up, warped mind, Prometheus was evil.” (CJ)

The depth of understanding is more significant since the implementation of the RISE feedback process.

Surprises

There were several surprises throughout implementation.  As already mentioned the conflict with student understanding in the Pre-Assessment generated unreliable data; consequently, it will not be used as a tool to determine growth of responses.  In addition, it was surprising to see that the most amount of growth between the genders tended to be associated with the males.  Another surprise was the data associated with students where Spanish is what is spoken at home.  Although the percentages were not as high as the native speakers, the increase was still greater.  There was some declines by a few percentage points here and there, and this surprised me, and has me re-evaluating the questions that the results generated-why did the boys show a slight decrease between trials two and three while the girls continued to exhibit growth after each trial?  In addition, I learned that I need to explain the RISE process before having students complete the pre-assessment.  This may help with the current unreliability of data housed within this assessment.  Students have commented on how they finally are understanding what I mean when I say what does this represent and how does this connect to the meaning.  They say that now that they know that their peers are going to RISE their responses, they feel it is important to put some thought behind their own responses.

Future Direction

I will continue working through Cycle 2.  I have already begun deciphering data.  I will incorporate the RISE song I created because I believe it will help my current participating students to remember the process to utilize.

I will continue utilizing the same process as outlined above for Cycle 2.  I feel that based on this data it is essential that all three trials be utilized to help create a mean of data and allow for possible confusing questions that students may struggle.  One other change I will implement is evaluating the data earlier to determine if there are any inconsistencies with the questions or peer responses.

Besides continuing with Cycle 2, I am going to continue gathering data from my Cycle 1 group.  I would like to evaluate if the growth continues as the school year continues.  It is my hope to rise all students to reach a 90 percent or higher in all areas of critical thought on the analytical rubric.