Home  | Introduction | Philosophy | Artifacts | Web Quest | Links | Reflective Statement  | Resume and Background Information

 

 

 

 

 

Improving Benchmark results: A study of 11th grade students using both teacher-guided and self-regulated learning.

 

 

 

 

 

 

 

 

 

by

 

Judith DeAntonellis

 

 

 

 

 

 

 

 

 

 

 

 

 

A Research Study

 

Submitted in partial fulfillment of the requirements for

Procedures and Evaluation in Research

of

The Educational Leadership Department

at Rowan University

Dr. Burton Sisco

December 9, 2009

 

 

 

Abstract

 

Judith DeAntonellis

Improving Benchmark results: A Study of 11th grade Students using both teacher-guided and self-regulated learning

2009

Dr. Burton Sisco

Procedures and Evaluation in Research

of

The Educational Leadership Department

 

            This study was conducted with the intention of exploring a variety of implementations provided to the 11th grade students enrolled in the Graduation Project 11 course at Pennsylvania Virtual Charter School (PAVCS) during the fall of 2009.  Though there were 259 students originally enrolled in the course, due to a high mortality rate and the fact that we were only examining those students that were in 11th grade, the number of students studied was actually 191.  The students in this study partook of two Benchmark tests provided by a program called Study Island.  It was anticipated that teacher direction would be valued in addition to self-regulated learning. Within these pages you will find the detailed results of this study. The students completed two Benchmark tests which were broken into five categories.  This study compares score results, grade results, and category results.  This study indicated little to no significant difference between the tests.  Ninety-five of the students that participated in both tests also partook of a survey.  The survey consisted of  36 Likert-type items using a five-point scale.  It also included a total of eight items that allowed for multiple or open ended responses.  I must forewarn the mortality rate (Change over of students in this environment) combined with outside influences (such as the other courses that have influenced the students’ learning) made the results of this study unreliable.  This was confirmed through the study via the open ended items.  More controlled and in depth study is necessary to draw any significant conclusions.  This study did draw attention to the various drawbacks of this study for those interested in expounding further.

 

 

 

 

Acknowledgments

            this work is dedicated to my husband, Dan and my son, Joe.  Both have supported me throughout my college career, despite the difficult time and money constraints that this endeavor has demanded.  Thank you both for your unwavering support and for understanding. Without you and your support, I would have never accomplished this.

            Thanks to the staff and students of Pennsylvania Charter School. Without your support and participation, this study would not have been possible.

            Finally, to Dr. Burton Sisco I owe my deepest gratitude for your dedication and willingness to answer my numerous questions well after class dismissed.  I could not have made it through without your support and advice.  You have inspired my interest in research for which I shall always be grateful. 

 

 

 

 

Table of Contents

 

 

Chapter                                                                                                                  Page

 

I

INTRODUCTION………………………………………………….

1

 

 

Statement of the Problem…………………………………

1

 

 

Purpose of the Study……………………………………...

1

 

 

Significance of the Study…………………………………

1

 

 

Assumptions and Limitations…………………………….

2

 

 

Operational Definitions…………………………………..

4

 

 

Research Questions……………………………………….

7

 

 

Overview of the Report…………………………………..

7

 

 

 

 

II

REVIEW OF LITERATURE………………………………………

9

 

 

A Brief Review of Self-Efficacy……………………….

9

 

 

The Consensus on the Value of Self-Regulated Learning..

10

 

 

The Few Discrepancies Regarding Self-Regulated Learning…………………………………………………...

12

 

 

Positive and Negative Aspects of Benchmark Testing…...

13

 

 

Summary of the Literature Review……………………….

15

 

 

 

III

METHODOLOGY…………………………………………………..

16

 

 

Context of the Study………………………………………

16

 

 

Population and Sample Selection…………………………

17

 

 

Instrumentation……………………………………………

18

 

 

Data Collection……………………………………………

20

 

 

Data Analysis……………………………………………..

20

 

 

 

 

IV

FINDINGS………………………………………………………….

22

 

 

Profile of the Sample……………………………………...

22

 

 

Analysis of the Data……………………………………….

25

 

 

Research Question 1………………………………………

25

 

 

Research Question 2………………………………………

36

 

 

 

 

V

SUMMARY, DISCUSSION, CONCLUSIONS, AND RECOMMENDATIONS…...........……………………………..

37

 

 

Summary of the Study…...........……………………….

37

 

 

Discussion of the Findings…...........…………………..

38

 

 

Conclusions………...........…………………………….

38

 

 

Recommendations for Further Practice and Research…………............……………………………

39

 

 

 

 

REFERENCES…………………………...........………………………………..

40

 

 

 

 

APPENDIX A:

Institutional Review Board Form….............…………………..

42

APPENDIX B:

Principal Permission and Subject Information Consent Form…………….............………………………………………

47

APPENDIX C:

Survey Instrument………...........…………………………...…..

51

APPENDIX D:

Disaggregate information from Study Island…...........………...........

59

APPENDIX E:

Correlation Charts and Tables.............................................................

62

 

list of tables

 

 

table                                                                                                                       Page

 

4.1.

Benchmark Data…………………….............……………………...

22

4.2

Survey Results Data......…………………………………………….

23

4.3

Benchmark Scores..............………………………………………...

25

4.4

Benchmark Grades......……………………………………………..

25

4.5

Benchmark Scores for Reporting Categories A and B......................

26

4.6

Benchmark Scores for Reporting Categories C and E......................

26

4.7

Benchmark Scores for Reporting Category D...................................

26

4.8

Likert Scale Questions from Survey................................................

27

4.9

Self-regulated goal setting.................................................................

30

4.10

Goal Setting Statements.....................................................................

31

4.11

Least helpful Interventions................................................................

32

4.12

Most helpful Interventions.................................................................

33

4.13

Student Recommendations................................................................

34

4.14

Influence of other classes..................................................................

35

4.15

Courses that Influenced test results...................................................

35

4.16

Other Courses Influences...................................................................

35

 

 

 

 

Chapter I

Introduction

Statement of Problem

            Benchmark testing has become a standard form of state standard test preparation.  Student self-regulated learning is considered essential for student success in such testing environments.  However, students need guidance.  Teacher-guided learning in a virtual environment is essential to student success.

Purpose of the Study

            The purpose of this study was to ascertain what if any of the methods used in the Graduation Project 11 course was/were beneficial to student learning and achievement in mathematics as demonstrated by comparing the results of Benchmark testing along with a student survey. I was interested in seeing if the various tools being utilized were beneficial to the students. I intended to use this study to assist with determining the future of math supports provided to the students.

Significance of the Study

This study examined self-regulated learning in combination with teacher-guided learning through the use of Benchmark testing as both a guide and a method of measurement.    The significance of this study was that it assumed that self-regulated learning alone does not ensure success.  As seen via many research studies, the consensus is that self-regulated learning is a key aspect of achievement for all students. However, this study examined the effects of teacher-guided learning in combination with a degree of self-regulated learning.   

 

Assumptions and Limitations

It was assumed that some if not all of the programs being used would benefit the students in some fashion.   This was also part of my bias, that the students would benefit from some if not all of the programs being used.  The environment was a virtual setting in which the students each worked from their homes.  It was assumed that the students were the participants.  In a virtual environment, it was the parents’ responsibility to ensure that the student was the one participating.  There was no definitive way of knowing whether there was parental coaching occurring during the Benchmark testing.  This also prohibited teacher observations of individual student responses and reactions.  There were approximately 50 students that did not complete the first of the two Benchmark tests.  Several of these students were special education.  The lack of evidence for the special education students skewed results. I included the results just the same.  According to the results of Benchmark I, only one student claimed to be a special education student.  This clearly limited this study. The disaggregated information that I was permitted to use for the study was taken from the generated report used in Study Island.  I was not permitted to ask such personal questions of the students.   The percentage of responses to the student survey was also a limitation.  With a standard a total of 44 questions to respond to, many high school students in this virtual setting did not complete the survey.  It was often difficult to get a large percentage of student participation in any given activity let alone something that does not impact their grades directly.  To offset this lack of motivation, I informed the students that completing the survey would influence the future lessons offered in this course.  Another limitation was the large turnover rate that occurs in a virtual charter school.  There were many students that completed the Benchmark I test that were not enrolled for completing the Benchmark II test.  Likewise, many students taking Benchmark II were not here to complete Benchmark I.  This distorted the findings of the disaggregated report provided by Study Island.  Each Benchmark test was not identical.  They are designed to randomly select questions from a base of 100 questions per question.  Each student was completing a unique to them test and the second Benchmark consisted of a different set of questions.  This limited the accuracy of the results.  As part of the schools initiative, I sent encouraging reminders for the students to prepare for the test and complete the test during the designated time frame. I also asked the Special Education Teachers speak with their families to ensure that the students would complete these tests during the required time frame as no extensions may be granted.  I was unable to locate studies that have a negative bias toward self-efficacy and/or self-regulated learning. All the studies I found were biased in that they firmly believe in the benefits of self-regulated learning and self-efficacy. I had a bias toward teacher-guided learning.  I firmly believed that students need guidance in addition to their own motivations to be truly successful.  I did my best to keep my bias out of this study, but I felt it necessary to recognize that it did exist and could potentially influence my reception of the results.

 

 

 

Operational Definitions

1.      11th grade students: These were students that were enrolled as 11th graders at the PAVCS High School in the Graduation Project 11 course during the fall of 2009. 

2.      Benchmark Testing: This is a form of formative test based on state standards in an attempt to replicate the general structure of state standardized tests.

3.      Blogging: A form of online journaling that is essentially considered a public journal.  However, in this study it was used with limitations.  Only those enrolled in the Graduation Project 11 course were able to access and view the Blogging area. This provided a safe environment for students to blog and become connected to each other. 

4.      Blue Ribbon: A ribbon appears next to the assignment that the students scored a 70% or above with a certain number of questions answered correctly. For example, a score of 70% with 7 out of 10 questions does not earn a Blue Ribbon in the scientific notation category.  One must score a 70% with at least 10 questions answered correctly to achieve a Blue Ribbon for that category.

5.      Cognition: The process of knowing, perceiving, etc. (Webster’s New World Dictionary and Thesaurus, p.117).

6.      Disaggregated: “Separate into parts: to separate something into its component parts, or break apart” ("Disaggregated," 2009). For the purposes of this study, it examined a variety of different categories: gender, race/ethnicity, Special Education, Title 1 recipients, English Language Learners, Socioeconomic status, and Gifted.

7.      Goal: “Aim: something that somebody wants to achieve” ("Goal," 2009). In this study goal was referring to the achievement of a score on the Benchmark tests or self determined achievements by the students when engaging in activities between the two tests. 

8.      Graduation Project 11: This was a course provided by Pennsylvania Virtual Charter School (PAVCS) that all 11th grade students were required to participate in.  It was worth .25 credit.  Overall, this course was to prepare the students for their Graduation Project 12 course.  Throughout the Graduation Project 11 course, students researched potential career choices, attended Elluminate class sessions, completed guided assignments in online programs such as Career Cruisin’ and Career Forward.  This was a course to assist students with assessing what careers they wished to endeavor upon in their futures.  Since all 11th grade students were enrolled in this course, it was a logical place to implement a support program to develop the students’ skills at completing state standardized tests that are given to all 11th grade students.  In this particular situation, they are participating in programs that include Benchmark testing to prepare them for the Pennsylvania System of School Assessment (PSSA).

9.      Metacognition: “The conscious awareness and frequent self-checking to determine if one’s learning goal has been achieved and, as necessary, selecting a more appropriate strategy to achieve that goal” (O’Neil & Abedi, 1996 as cited by Malpass, O’Neil, & Hacevar, 1999, p.1).

10.   Mortality: "When subjects systematically drop out of or are lost from the study and their absence affects the results" (McMillan, 2008, p.223).  In this study it refers to the subjects who participated in one part of the study but not another or those that did not participate at all.

11.  Reliability: "Consistency of scores" (McMillan, 2008, p.149). In this study, the influences of other courses and the lack of a significant change from Benchmark 1 to Benchmark II results affected the reliability of this study.  The demographics and the grades were unreliable due to the mortality rate at this school.  

12.  Self-Efficacy: “Personal judgment of one’s performance capabilities in specific situations that may contain ambiguous, unpredictable, and stressful features”(Schunk, 1985, p.1).

13.  Self-Regulated Learning Strategy: Setting goals, rewards, punishments, methods for developing knowledge of given subject, assessment and evaluation by the student (Zimmerman & Martinez, 1986). In this study, students spent one day per week determining their weaknesses and focused on developing those skills, after participating in a teacher assigned math game, the students wrote a reflective blog about their experiences each week.  

14.  Study Island: A program geared toward State Standards that provides a form of benchmark testing and assignments that are geared to specific standards. In this study, they were preparing for the Pennsylvania System of School Assessment (PSSA).  Students completed the assignments in game mode, test mode, classroom response systems or printable worksheets. If they did well they could earn a Blue Ribbon for the assignment.  

15.  Teacher-guided: In this study, it was referring to materials provided by the teacher that the students were to use in developing their knowledge.  More specifically, the teacher provided links to the definitions of the math words of the week, assignment to complete a Study Island Blue Ribbon assignment each week, and math games provided through websites posted by the teacher each week.

16.  Validity: " The extent to which inferences are appropriate and meaningful" (McMillan, 2008, p.144).  In this study, the mortality rate and influences from other courses made the results in this study invalid.

Research Questions

This study sought to answer these questions:

            1. Does the use of Benchmark testing as a guide assist students with achieving goals when testing the second time?

            2. Which methods of learning are most beneficial—teacher-guided or self-regulated?

 

Overview of the Report

            In chapter II, there is a literature review.  The reports reviewed herein are biased towards self-efficacy and self-regulated learning.  The benchmark testing researchers appear to have a bias toward the assumptions of their studies.  Oddly, both Benchmark studies had control groups in which they did not ascertain the programs used to support student achievement in mathematics.  Therefore, their results were inconclusive.  The self-efficacy and self-regulated learning studies are determined that there is a flaw in their study if their results do not support their assumptions. 

            In chapter III, the methodology and processes used to conduct the study are detailed. The key aspects covered herein are: the environment of this study; the population and sampling used in this study; the instrument of Benchmark testing results and a student survey, how these instruments are administered, and an analysis of the findings. 

            In chapter IV, the detailed results of this study are presented.  This is the area that most directly addresses the research questions posed in chapter one.  Statistical data will dominate over the narrative data used to summarize the results. 
            In chapter V, there is a summary and discussion about the results of this study.  This includes conclusions drawn from the study and recommendations for future studies. 

 

 

 

 

Chapter II

Review of Literature

A Brief Review of Self-Efficacy

According to studies by Schunk and Bandura (as cited in Schunk,1985), learning-disabled students have a lower sense of self-efficacy causing negative emotional responses, poor effort, and lower skill achievement. Malpass, O’Neil, and Hacevar (1999) support this concept in their determination that high achieving students have a greater sense of self-efficacy than lower level students.

Self-Efficacy greatly affects student performance (Schunk 1985).  Schunk (1985) hypothesizes that self-efficacy impacts student “choice of activities, effort expenditure, perseverance, and task accomplishments”(p.307)  According to Zimmerman and Pons (1986), low achieving students were twice as likely to make will power statements as high achieving students were. This implies low achieving students focus more on effort and perseverance than on achievement and understanding. According to Bandura ( as cited in Malpass, O’Neil, & Hacevar, 1999), self-efficacy affects motivation through the goals people set, effort they exert, tendency toward perseverance, and hardiness when faced with failure.  Malmivuori (2006) found that students’ high self-efficacy positively affects math performance while students’ low self-efficacy negatively affects math performance.       

 

 

The Relationship Between Self-Regulated Learning and Self-Efficacy

            Setting specific goals leads to higher achievement than no goals or goals that are not specific, “such as ‘do your best’” (Schunk, 1985, p. 308).  According to Schunk (1985), once the student has achieved the set goal, the student’s sense of self-efficacy is validated.  Schunk’s study (1985) compares students who have set their own goals with those whose goals have been set for them. Schunk’s study (1985) demonstrates that students setting their own goals developed a higher sense of self-efficacy.  Therefore, this study supports the theory that self-efficacy is not a stagnant level determined by past experiences, but that self-efficacy can be developed through the use of self-regulation strategies such as goal setting. Malpass, O’Neil, and Hacevar (1999), cite a variety of studies demonstrating that there is a correlation between self-efficacy and self-regulation.  Some of these studies indicate as the above studies, that self-efficacy is positively affected by self-regulation. There is one study by Garcia and Pintrich (as cited by Malpass et al, 1999) that demonstrates how self-regulated learning is in turn affected by self-efficacy. Self-efficacy provides a positive association to self-regulation (Malpass et al., 1999). Both Zimmerman and Schunk support self-regulation promoting a more positive sense of self-efficacy (as cited by Fuchs, Fuchs, Prentice, Burch, Hamlett, Owen, & Schroeter, 2003).

The Consensus on the Value of Self-Regulated Learning

            Schunk (1985) ascertains that student’s involvements in goal setting may be more beneficial for low achieving students than for those who are more confident. According to Bandura, Schunk, and Zimmerman (as cited by Zimmerman & Pons, 1986), “Academic achievement is one realm where self-regulated learning processes are assumed to be crucial” (p. 615). Zimmerman and Pons (1986) go on to ascertain that school achievement in higher grade levels is severely impacted by self-regulation due to the amount of private study required. The study conducted by Zimmerman and Pons (1986) found 91% of the students could be correctly classified (as either high-level or low-level learners) using their self-regulation strategies.  High-achieving students “frequently” use self-regulated learning strategies (Zimmerman & Pons, 1986).  By comparison, low-achieving students “occasionally” use self-regulated learning strategies (Zimmerman &Pons, 1986). The fact that low-level learners used more will power statements rather than self-regulated strategies demonstrates the difference in learning strategies of high-level and low-level learners (Zimmerman & Pons 1986). According to Malpass, O’Neil, and Hacevar (1999), gifted children have better metacognition and retention strategies than average students do. Low-level learners see ability as a stagnant and immovable trait, but high-level learners see their capacity for learning as something that can change and increase (Malpass et al., 1999).  According to Cervone (as cited by Fuchs et al., 2003), self-regulated learning ought to advance learning and students are motivated by setting goals to achieve their objectives. Fuchs et al. (2003), find that self-regulated learning  provided reliably stronger achievement when used with “problem solving transfer treatment” when compared to the control group (p.313). This study (Fuchs et al., 2003) also found a correlation between the students in the combined self-regulated and “problem solving transfer treatment” group and their sense of effort (p.313).  In a questionnaire, the students in this combined group felt more strongly that they had made a significant effort to be successful in math (Fuchs et al., 2003).  According to Boekarts and Turner et al. (as cited by Malmivuori, 2006), intense emotions correlate to student self-assessments as regards goals and effort in math. Preconceptions of math due to past personal experiences influence students constructed beliefs regarding the necessity and usefulness of mathematics (Malmivuori, 2006). The study conducted by Kramarski and Zoldan (2008), provides results indicating that students using a combination of results analysis and self analysis were more successful than groups using either one or the other by itself. 

The Few Discrepancies Regarding Self-Regulated Learning

            Schunk’s (1985) study set upper and lower limits for goal setting for learning disabled students. Schunk (1985) cites Robbins and Harway; and Tollefson et al. for the assertion that learning-disabled students determine inappropriate goals and may not utilize performance details as a method to set an appropriate goal. Unrealistic goals, whether too high or too low undermine a students ability to achieve as well as their sense of self-efficacy (Schunk 1985).  According to Tollefson et al. (as cited by Schunk, 1985), students may need to be trained in how to set goals.  Zimmerman and Pons (1986) indicate that further study of these issues is worthwhile. Self-evaluation is the one area of self-regulation that does not correlate to student achievement. “Bandura suggested that self-regulatory skills are meaningless if students cannot apply themselves in a persistent manner in the face of difficulties, distractions, and stress, and that ‘self-directed learning requires motivation as well as cognitive and metacognitive strategies (Malpass, O’Neil, & Hacevar, 1999, 2).  Malpass et al.  (1999) did not find a significant correlation between success in high-stakes mathematics assessment and self-regulation.  “There is little consensus among researchers as to how self-regulation should be reported when self-report scales are used… (Malpass et al., 1999, p. 7). Malpass et al. (1999) found their results to be confounding regardless of the differences in measurement strategies.  Unexpectedly, they discovered that there is no statistically significant connection between test apprehension and self-regulation (Malpass et al.,1999). In this study, there is a greater connection between self-efficacy and achievement than self-regulation and achievement (Malpass et al., 1999).  Contrary to Schunk’s study (1985), this study (Malpass et al., 1999) found no statistically significant relationship between goal setting and self-efficacy. Students with learning difficulties struggled at determining how well they used a strategy.  Therefore, the results for low achievers may not be as reliable. The study by Fuchs et al. (2003) provided mixed results. Although the students using self-regulated learning did score higher than those in the control group did, there was little change when comparing the pretest to the posttest.  From Schunk’s (as cited by Fuchs et al., 2003) perspective, low achieving students have a greater potential for inaccurately recording their performance.  The results from Fuchs et al.’s (2003) study reported no statistically significant growth. The self-regulatory aspects of Malmivuori’s (2006) study show a greater correlation to motivation than ability to use metacognitive strategies.  There are not enough studies to properly represent the positive and negative aspects of using multiple metacognitive approaches (Kramarski & Zoldan, 2008).

Positive and Negative Aspects of Benchmark Testing

            According to the study conducted by the Institute of Educational Sciences [IES] (2007), after one year, there is no statistically significant student achievement occurring from the use of Benchmark testing.  According to a U.S Department of Education report (as cited by IES, 2007), benchmark testing is often used to assess student mastery of state standards.  Black and Williams (as cited by the IES, 2007) determined that formative assessments have a statistically significant positive impact on student learning and that teachers and students use such tests to effectively determine achievements goals.  However, during the study conducted by IES (2007), it was determined that there is not a statistically significant improvement in student achievement by using benchmark testing after one year of implementation. Though there is a difference between the control group and the program group, it is so minimal it could as likely be due to chance as to the benchmark program studied (IES, 2007).  The main reason for this minimal difference is that there are no data on the form of benchmark assessment the comparative schools are using (IES, 2007).  It is assumed that all schools are implementing some form of benchmark testing (IES, 2007). Therefore, the IES (2007) study demonstrates that there is likely a minimal difference in which program of benchmark testing a school uses.  Also of note, in the IES study, is that there was only one year of implementation.  Further study in this area clarifying the control/comparison school’s practices and spanning for several years would be more comprehensive (IES, 2007). Achievement in mathematics will occur not from the existence of Benchmark tests but from the fashion in which they are used (IES, 2007). To this effect, the Educational Testing Service’s Manifesto (as cited by Sacks, 2009) states, “assessments must evolved from being isolated occasional events attached to the end of teaching to becoming an ongoing series of interrelated events that reveal changes in students learning over time” (p.1). The key aspect that is mentioned in this brief is that the interaction and instruction that occur between assessments is what will bring about students achievement (Sacks, 2009). The use of the Galileo program is examined through Sack’s brief.  There is student-level positive relation to success in Galileo and in other external assessments (Sacks, 2009).  However, it has not been examined at the school-level by the MCAS (Sacks, 2009). Meeting the Benchmark standards correlated to meeting the MCAS standard on a range of 80-90% (Sacks, 2009). A key aspect of this brief is that teacher’s used the assessments to guide instruction (Sacks, 2009). There is a statistically significant difference between students with teachers that had high implementation(defined as 66 percentile or above in the use of benchmark data guiding instruction) of this practice by comparison to teachers that had a low implementation (defined as 33 percentile or below for the implementation of benchmark data to guide instruction)(Sacks, 2009). There is no significant difference between school’s using this program and schools in the control group.  This is thought as in the previous study to be due to the lack of knowledge about the programs being implemented in the control schools. 

Summary of the Literature Review

            Although there is a great deal of study in regards to self-regulation and self-efficacy, there appear to be no studies that seek to identify negative aspects of self-regulation and self-efficacy.  All studies indicate a bias on the side of the researchers that assumes all agree that self-regulation and self-efficacy are undeniably relevant and effective methods for learning.  Even in studies where the results clearly contradict the significance of self-regulation and self-efficacy in student achievement, the researches seem unable to grasp the significance of their results.  More study from non-biased or from those that are against self-regulation, is needed to truly have a strong understanding of this subject matter. 

Benchmark testing appears to have some relevance as regards individual achievements for students.  However, it is difficult to compare to a program without such preparations, as these preparations are standard practice in most schools.  More research is needed to fully ascertain the benefits and shortcomings of Benchmark testing. 

 

 

 

CHAPTER III

METHODOLOGY

Context of the Study

            The study was conducted at Pennsylvania Virtual Charter School based out of Norristown, PA.  Although the school was based out of Norristown, PA, the students were from all across the state.  This makes the demographic details difficult to specify.  The large turnover rate in this particular environment also made any specific declaration difficult.  According to the Pennsylvania Department of Education (PDE) Charter Annual Report (2008), “These students emanate from diverse ethnic, racial, (sic) socio-economic backgrounds” (p. 3).

            This charter began on September 4, 2001 as a K thru 8 school.  It expanded to a K thru 12 school.  The first graduating class was the class of 2009.  The essence of this charter is demonstrated via its diamond model.  There are four main components: a certified teacher, family support coordinator, learning coach (usually the student's parent), and the K-12 curriculum.  According to the PDE Charter Annual Report, as of October 1, 2007, there were 3740 students enrolled in the then K-11 school. “This school year, PAVCS had students enrolled from 456 of the 501 school districts in the Commonwealth. The schools FTE for this year totaled 3302.86 for regular education and 357.95 for special education” (PDE Charter Annual Report, p. 3). The large turnover rate make year to year comparisons difficult.  This is listed as one of the challenges that the school faces in the PDE Annual Charter Report. 

            The Pennsylvania Virtual Charter School utilizes Blackboard Academic Suite and Study Island as two of the online components.  Students were also required to attend classes in virtual meeting rooms through a program called Elluminate.  The school provided a subsidy to all students to ensure that they had internet access. All students had computers, printers, ink, and paper provided by the school.   

Population and Sample Selection

            The target population was all 11th grade students in the Commonwealth of Pennsylvania.  The available population was all 11th grade students enrolled at Pennsylvania Virtual Charter School that participated in both Benchmark I and Benchmark II tests.  The convenience sample was determined because of personal involvement with reminding the students to complete the Benchmark tests as part of my roll in the Graduation Project 11 team.  The Graduation Project 11 course is taught by a group of four: two teachers and two guidance counselors.  The guidance counselors and one of the two teachers, teach for a span of one to two months at a time.  This means that they create the lesson plans that the students utilized and provided the Elluminate instruction for the materials connected to determining a career choice.  The fourth teacher is in charge of sending reminders to students to complete the Benchmark tests and creating supplemental materials and assignments for the students to complete between each of the tests.  The focus is on the Math Benchmark test as the students as a whole have done poorly on the Pennsylvania System of School Assessment consistently each year.  Two-hundred-and-twenty-five students participated in Benchmark I.  Two-hundred-and-twenty-four students participated in the Benchmark II test.  One-hundred-and sixteen students participated in the Benchmark Survey.  Only 95 of those also completed the Benchmark I and Benchmark II tests.

Instrumentation

            The instruments to measure student achievement were the Benchmark I test, Benchmark II test, and the Benchmark survey.  The Benchmark I test and the Benchmark II test were devised by school personal using the Study Island program. Each test was individualized and consisted of varying questions, which makes them consistent with the Pennsylvania System of School Assessment test after which they were designed.  The survey was self-designed to be specific to this unique school environment and program of study.  It consists of 44 items.  These items were selected considering the course curriculum and programs used to assist students with developing their math skills to be more successful with their Benchmark testing.  The items were geared to the specific assignments provided to the students.  The items referenced the usefulness and engaging qualities of each activity.  The activities were Study Island assignments, assigned games, student selected games, writing blogs, and completing Blue Ribbon assignments.  To determine content validity, the survey was distributed to five students in the Graduation Project 11 course, who were in an accelerated program.  They were actually 10th grade students that intend to graduate early. They were asked to complete a test run and provide feedback regarding this survey. Only one of the five actually completed and commented on the survey. It was also sent to the lead teacher, who is the supervisor of the teacher conducting this study, and the other teacher involved in the Graduation Project 11 course.   

            This survey was designed to measure student perceptions of the programs of study used to support them between the two Benchmark tests.  The survey questions were designed to ascertain whether the student self-regulated learning or the teacher-guided instruction was more beneficial.  Reliability coefficients are provided in Appendix E.  There appears to be moderate to moderately strong correlations between all Study Island and Blue Ribbon items. There is a weak to moderately strong correlation between items that relate to self-regulated learning. There is a moderate to moderately strong correlation between the items that relate to writing blogs.  Ironically, these items have a weak correlation to the item concerning which intervention was least helpful.  It is ironic as the predominate item was writing blogs. 

            The benchmark survey (Appendix C) consisted of two sections.  Section one determined the student and grade level.  The student information was to ensure that only one response per student was evaluated. The second section consisted of a combination of Likert scale and short explanation/answer items. These items were designed to evaluate the students’ perceptions of what course materials were engaging and what course materials were beneficial to the development of their math skills.

            Following approval from the Institutional Review Board of Rowan University (Appendix A), a pilot test was administered to five students that were enrolled in Graduation Project 11 but not 11th grade students.  Five 10th grade students that were enrolled in the Graduation Project 11 course were given the test to complete a test run and provide feedback regarding this survey.  Only one student responded with minimal input. She did indicate that the items in the middle seemed redundant.  I therefore, rearranged the order of the items to avoid the sense of redundancy.

Data Collection

            Permission was granted from the Principal of the School to survey the students (Appendix B). The survey (Appendix C) was posted on the main course page on Blackboard and emailed to all students of the Graduation Project 11 course through the Blackboard e-mail system.  It was distributed in November, 2009, after the students completed the Benchmark II test.  Other than to ascertain that only one response was received per student and that only the responses from 11th grade students were examined, no other identifying information was collected. All identifying information was replaced with Student A, Student B and so forth.  The only incentive that could be realistically provided was that the students knew that their responses would guide future instruction in this course.

            The results of the Benchmark I and Benchmark II tests were also collected and compared.  The disaggregated results were completed using the Study Island Executive Summary Report.  This causes all details to be provided whether the same students were enrolled for each test or not to be compared.  The other results were collected from Study Island and then evaluated through the Statistical Package for the Social Sciences (SPSS) software. Only the results of students that participated in both tests were examined.   

Data Analysis

`           The independent variables were collected through the Study Island program and the first part of the survey.  The Study Island program provides the disaggregated results of the reports run.  I do not have access to the details on how this information was achieved, but it was clear that it was a form of survey that participants could choose to participate.  The survey asked only for grade level, first name, last initial and the last three digits of the students school ID. All information was replaced with non-descript names such as Student A, Student B and so forth.  The grade level is the one independent variable retained so that only the 11th grade students were examined. The remaining items are the dependant variables of the study.  The students were asked 44 questions. The results of these questions were analyzed using the Statistical Package for the Social Sciences (SPSS) software.   Charts were used in Appendix E to demonstrate the reliability coefficients.  Frequency tables were used to demonstrate the results of the data.  There should be no impact of the independent variables on the dependant variables as that was not the sort of information that was collected.  The SPSS system was used to analyze the data. Excel and Microsoft word were used to create the table and charts. 

 

Chapter IV

Findings

Profile of the Sample

            The subjects of this study were students that participated in the Graduation Project 11 course at Pennsylvania Charter School (PAVCS).  Details of all 191 11th-grade students that participated in both Benchmark tests were included in those reports.  A survey was also distributed to the students of this course.  Ninety-five of the 11th grade students participated in the survey, yielding a return of 50%.  The demographics were provided through the Study Island program.  They considered all 11th grade students regardless of the mortality rate.  The details are provided in Appendix D.

            Table 4.1 contains data on the benchmark tests.  It was broken down into the categories of the test along with the final grade and score. 

Table 4.1. Benchmark Data (N=191)                                                             

Category

n

M

Mdn

SD

Benchmark 1 score

191

1.87

2.00

.911

Benchmark 2 score

191

1.81

2.00

.864

Benchmark 1 grade

191

1.92

2.00

.994

Benchmark 2 grade

191

2.11

2.00

1.068

Benchmark 1 Numbers and Operations (Reporting Category A)

191

3.21

3.00

1.095

Benchmark 2 Numbers and Operations (Reporting Category A)

191

3.32

3.00

.984

Benchmark 1 Measurement (Reporting Category B)

191

3.47

3.00

1.070

Benchmark 2 Measurement (Reporting Category B)

191

3.19

3.00

1.199

Benchmark 1 Geometry (Reporting Category C)

191

3.21

3.00

1.282

Benchmark 2 Geometry (Reporting Category C)

191

3.47

3.00

1.424

Benchmark 1 Algebraic Concepts (Reporting Category D)

191

6.33

6.00

2.543

Benchmark 2 Algebraic Concepts (Reporting Category D)

191

6.10

6.00

2.938

Benchmark 1Data Analysis and Probability (Reporting Category E)

191

3.51

4.00

1.218

Benchmark 2 Data Analysis and Probability (Reporting Category E)

191

3.84

4.00

1.122

 

Table 4.2 provides the data related to the survey results.

Table 4.2. Survey Results Data (N=95)

Survey Question

n

M

Mdn

SD

The goal of a 10% increase between the Math Benchmark I and the Math Benchmark II test was too difficult to accomplish

95

2.86

3.00

1.027

Writing a Blog about my experiences helped me develop my math skills knowledge.

95

2.55

3.00

1.128

Using sites of my choosing was beneficial in developing my math skills.

95

3.41

4.00

1.171

Using the Benchmark I test as my guide was the most beneficial

95

3.47

4.00

1.050

I set goals for myself each week for *choose all that apply

95

2.673

1.200

2.1630

Writing a Blog each week was the most beneficial

95

2.39

2.00

1.065

Has your work in other classes affected your skills?

95

1.21

1.00

.410

If you answered yes to the above question, which course affected your skills the most?

95

1.89

1.00

1.216

If you selected other in the above question, which other course affected your skills the most?

14

3.29

3.00

1.773

Choosing what skills I worked on gave me a sense of control. 

95

3.87

4.00

.914

Playing games of my choosing was the most beneficial

95

3.26

3.00

1.187

The math words of the week were not beneficial in developing my understanding.

95

2.82

3.00

1.329

The Study Island exercises were mentally engaging

95

3.64

4.00

1.148

Completing the Blue Ribbon assignments each week increased my confidence in my math skills

95

3.57

4.00

1.235

Exploring the math words of the week via the links provided in Graduation Project 11 expanded my understanding of the these terms.

95

3.24

3.00

1.218

Playing games posted on the external links page was the most beneficial

93

2.83

3.00

1.138

Picking the skills I focused on each week helped me to develop my math skills.

95

3.54

4.00

1.099

The goal of a 10 % increase between the Math Benchmark I and the Math Benchmark II was a good goal for me.

95

3.27

3.00

1.189

 I struggled with understanding my weakest and strongest aspects when looking at the results of my Math Benchmark I results.

95

2.94

3.00

1.109

Using the sites provided on the external links pages was mentally engaging.

95

2.91

3.00

.979

Table 4.2. Survey Results Data continued

 

 

 

 

Survey Question

n

M

Mdn

SD

Playing games posted on the main page of Graduation Project 11 was the most beneficial

95

2.69

3.00

1.053

It was helpful to use the Math Benchmark I test to guide my preparations for the Math Benchmark II test.

95

3.38

3.00

1.103

The game sites posted each week on the main page of Graduation Project 11 strengthened my math skills.

95

2.82

3.00

1.041

Knowing the results of the Math Benchmark I test did not help me extend my math knowledge

95

2.91

3.00

1.221

 I watched the math videos each week even though the course did not require it.

95

2.18

2.00

1.062

Working on Study Island in the test mode was the most beneficial

95

3.89

4.00

1.162

Writing a blog about my math experiences each week was mentally engaging

95

2.42

2.00

1.145

Study Island exercises helped me to analyze math

95

3.63

4.00

1.102

I would have preferred to set my own goal for the Benchmark II test

95

3.46

3.00

1.183

Completing the Blue Ribbon assignments each week extended my math knowledge

95

3.63

4.00

1.092

Writing a Blog about my experiences helped me retain my math skills knowledge.

95

2.53

3.00

1.090

It was easy to figure out my strengths and weaknesses using the Math Benchmark I test as my guide.

95

3.45

3.00

1.008

Watching the math videos each week helped me with math comprehension

95

2.66

3.00

1.182

The goal of a 10 % increase between the Math Benchmark I and the Math Benchmark II was too easy to accomplish.

95

2.27

2.00

1.036

Completing the Blue Ribbon assignments each week was mentally engaging

95

3.61

4.00

1.142

The math words of the week helped me develop my understanding of math.

95

2.83

3.00

1.155

The game sites posted each week on the main page of Graduation Project 11 were mentally engaging.

95

2.67

3.00

1.015

The sites provided on the external links page extended my math skills knowledge.

95

2.86

3.00

1.038

The Study Island exercises helped me comprehend math.

95

3.77

4.00

1.106

Being able to use sites of my choosing to develop my math skills gave me a sense of control.

95

3.60

4.00

1.115

Which of the things used in Graduation Project 11 assisted you the least with being successful in the Benchmark II test?

82

8.51

10.50

3.827

 

Analysis of the Data

Research Question 1: Does the use of Benchmark testing as a guide assist students with achieving goals when testing a second time?

            By reviewing the results provided in Tables 4.3 and 4.4  there was little difference in the overall scores and grades of the participants. Table 4.3 shows a decrease in overall scores.  The mean scores actually drop a little between Benchmark I and Benchmark II. When referring to Table 4.4 it appears that the overall grades do increase. However, that increase was via comparison to how other students throughout the state scored.  Since the Study Island grades ignore mortality rates, these results are not reliable and any inferences made would be invalid.

 

Table 4.3. Benchmark Scores

 

Test I

 

Test II

Score

f

P

 

f

P

0% thru 43.30%

87

45.5

 

87

45.5

46.70% thru 56.70%

48

25.1

 

60

31.4

60% thru 83.30%

50

26.2

 

38

19.9

86.70% thru 100%

6

3.1

 

6

3.1

Total

191

100

 

191

100

 

 

Table 4.4. Benchmark Grades

 

Test I

 

Test II

Grade

f

P

 

f

P

Below Basic

90

47.1

 

72

37.7

Basic

35

18.3

 

53

27.7

Proficient

56

29.3

 

39

20.4

Advanced

10

5.2

 

27

14.1

Total

191

100

 

191

100

 

 

 

 

 

 

 

 

Table 4.5. Benchmark Scores for Reporting Categories A and B

 

 

Reporting Category A--

Numbers and Operations

 

Reporting Category B--

Measurement

 

Test I

 

Test II

 

Test I

 

Test II

Score

f

P

 

f

P

 

f

P

 

f

P

0%

9

4.7

 

5

2.6

 

7

3.7

 

14

7.3

25%

45

23.6

 

36

18.8

 

27

14.1

 

43

22.5

50%

60

31.4

 

62

32.5

 

64

33.5

 

64

33.5

75%

51

26.7

 

68

35.6

 

56

29.3

 

33

17.3

100%

26

13.6

 

20

10.5

 

37

19.4

 

37

19.4

Total

191

100

 

191

100

 

191

100

 

191

100

 

Table 4.6. Benchmark Scores for Reporting Categories C and E

 

Reporting Category C--

Geometry

 

Reporting Category E--

Data Analysis and Probability

 

Test I

 

Test II

 

Test I

 

Test II

Score

f

P

 

f

P

 

f

P

 

f

P

0%

14

7.3

 

16

8.4

 

13

6.8

 

7

3.7

20%

45

23.6

 

33

17.3

 

28

14.7

 

18

9.4

40%

61

31.9

 

55

28.8

 

42

22

 

36

18.8

60%

38

19.9

 

41

21.5

 

72

37.7

 

72

37.7

80%

23

12

 

25

13.1

 

29

15.2

 

53

27.7

100%

10

5.2

 

21

11

 

7

3.7

 

5

2.6

Total

191

100

 

191

100

 

191

100

 

191

100

 

Table 4.7. Benchmark Scores for Reporting Category D

Reporting Category D--Algebraic Concepts

 

Test I

 

Test II

Score

f

P

 

f

P

0%-25%

52

27.3

 

33

17.2

33.3%-50%

87

45.5

 

78

40.8

58.3%-75%

36

18.9

 

61

32

83.3%-100%

16

8.4

 

19

9.9

Total

191

100

 

191

100

 

 

 

Table 4.8. Likert Scale Questions from Survey

(1=Strongly Disagree, 2= Disagree, 3=Neutral, 4= Agree, 5= Strongly Agree)

 

Survey Question

 

1

2

3

4

5

Total

1

The goal of a 10% increase between the Math Benchmark I and the Math Benchmark II test was too difficult to accomplish

f

10

20

45

13

7

95

 

P

10.5

21.1

47.4

13.7

7.4

100

 

 

 

 

 

 

 

 

 

2

The goal of a 10 % increase between the Math Benchmark I and the Math Benchmark II was a good goal for me.

f

11

9

33

27

15

95

 

P

11.6

9.5

34.7

28.4

15.8

100

 

 

 

 

 

 

 

 

 

3

I would have preferred to set my own goal for the Benchmark II test

f

9

5

36

23

22

95

 

P

9.5

5.3

37.9

24.2

23.2

100

 

 

 

 

 

 

 

 

 

4

The goal of a 10 % increase between the Math Benchmark I and the Math Benchmark II was too easy to accomplish.

f

27

26

34

5

3

95

 

P

28.4

27.4

35.8

5.3

3.2

100

 

 

 

 

 

 

 

 

 

5

Writing a Blog about my experiences helped me develop my math skills knowledge.

f

20

27

28

16

4

95

 

P

21.1

28.4

29.5

16.8

4.2

100

 

 

 

 

 

 

 

 

 

6

Writing a Blog each week was the most beneficial

f

22

30

31

8

4

95

 

P

23.2

31.6

32.6

8.4

4.2

100

 

 

 

 

 

 

 

 

 

7

Writing a blog about my math experiences each week was mentally engaging

f

24

29

24

14

4

95

 

P

25.3

30.5

25.3

14.7

4.2

100

 

 

 

 

 

 

 

 

 

8

Writing a Blog about my experiences helped me retain my math skills knowledge.

f

21

23

34

14

3

95

 

P

22.1

24.2

35.8

14.7

3.2

100

 

 

 

 

 

 

 

 

 

9

Using sites of my choosing was beneficial in developing my math skills.

f

7

14

25

31

18

95

 

P

7.4

14.7

26.3

32.6

18.9

100

 

 

 

 

 

 

 

 

 

10

Choosing what skills I worked on gave me a sense of control. 

f

2

2

28

37

26

95

 

P

2.1

2.1

29.5

38.9

27.4

100

 

 

 

 

 

 

 

 

 

11

Playing games of my choosing was the most beneficial

f

10

12

31

27

15

95

 

P

10.5

12.6

32.6

28.4

15.8

100

 

 

 

 

 

 

 

 

 

12

Picking the skills I focused on each week helped me to develop my math skills.

f

6

9

26

36

18

95

 

P

6.3

9.5

27.4

37.9

18.9

100

 

 

 

 

 

 

 

 

 

13

Being able to use sites of my choosing to develop my math skills gave me a sense of control.

f

4

9

34

22

26

95

 

P

4.2

9.5

35.8

23.2

27.4

100

Table 4.8. Likert Scale Questions from Survey Continued

(1=Strongly Disagree, 2= Disagree, 3=Neutral, 4= Agree, 5= Strongly Agree)

 

 

Survey Question

 

1

2

3

4

5

Total

14

Using the Benchmark I test as my guide was the most beneficial

f

6

8

30

37

14

95

 

P

6.3

8.4

31.6

38.9

14.7

100

 

 

 

 

 

 

 

 

 

15

Knowing the results of the Math Benchmark I test did not help me extend my math knowledge 

f

15

20

29

21

10

95

 

P

15.8

21.1

30.5

22.1

10.5

100

 

 

 

 

 

 

 

 

 

16

It was easy to figure out my strengths and weaknesses using the Math Benchmark I test as my guide.

f

5

7

37

32

14

95

 

P

5.3

7.4

38.9

33.7

14.7

100

 

 

 

 

 

 

 

 

 

17

It was helpful to use the Math Benchmark I test to guide my preparations for the Math Benchmark II test.

f

7

9

36

27

16

95

 

P

7.4

9.5

37.9

28.4

16.8

100

 

 

 

 

 

 

 

 

 

18

I struggled with understanding my weakest and strongest aspects when looking at the results of my Math Benchmark I results.

f

12

17

39

19

8

95

 

P

12.6

17.9

41.1

20

8.4

100

 

 

 

 

 

 

 

 

 

19

Using the sites provided on the external links pages was mentally engaging

f

8

23

37

24

3

95

 

P

8.4

24.2

38.9

25.3

3.2

100

 

 

 

 

 

 

 

 

 

20

Playing games posted on the external links page was the most beneficial

f

14

19

37

15

8

93

 

P

14.7

20

38.9

15.8

8.4

97.9

 

 

 

 

 

 

 

 

 

21

The game sites posted each week on the main page of Graduation Project 11 strengthened my math skills.

f

12

21

38

20

4

95

 

P

12.6

22.1

40

21.1

4.2

100

 

 

 

 

 

 

 

 

 

22

Playing games posted on the main page of Graduation Project 11 was the most beneficial

f

15

22

39

15

4

95

 

P

15.8

23.2

41.1

15.8

4.2

100

 

 

 

 

 

 

 

 

 

23

The game sites posted each week on the main page of Graduation Project 11 were mentally engaging.

f

14

23

42

12

4

95

 

P

14.7

24.2

44.2

12.6

4.2

100

 

 

 

 

 

 

 

 

 

24

The sites provided on the external links page extended my math skills knowledge.

f

12

17

43

18

5

95

 

P

12.6

17.9

45.3

18.9

5.3

100

 

 

 

 

 

 

 

 

 

25

The math words of the week were not beneficial in developing my understanding.

f

21

17

28

16

13

95

 

P

22.1

17.9

29.5

16.8

13.7

100

 

 

 

 

 

 

 

 

 

Table 4.8. Likert Scale Questions from Survey Continued

(1=Strongly Disagree, 2= Disagree, 3=Neutral, 4= Agree, 5= Strongly Agree)

 

 

Survey Question

 

1

2

3

4

5

Total

26

Exploring the math words of the week via the links provided in Graduation Project 11 expanded my understanding of the these terms.

f

11

13

28

28

15

95

 

P

11.6

13.7

29.5

29.5

15.8

100

27

 I watched the math videos each week even though the course did not require it.

f

31

28

27

6

3

95

 

P

32.6

29.5

28.4

6.3

3.2

100

 

 

 

 

 

 

 

 

 

28

The math words of the week helped me develop my understanding of math.

f

14

22

33

18

8

95

 

P

14.7

23.2

34.7

18.9

8.4

100

 

 

 

 

 

 

 

 

 

29

Watching the math videos each week helped me with math comprehension

f

19

22

34

12

8

95

 

P

20

23.2

35.8

12.6

8.4

100

 

 

 

 

 

 

 

 

 

30

The Study Island exercises were mentally engaging

f

5

10

25

29

26

95

 

P

5.3

10.5

26.3

30.5

27.4

100

 

 

 

 

 

 

 

 

 

 

Working on Study Island in the test mode was the most beneficial

f

5

8

15

31

36

95

31

P

5.3

8.4

15.8

32.6

37.9

100

 

 

 

 

 

 

 

 

 

 

Study Island exercises helped me to analyze math

f

5

8

27

32

23

95

32

P

5.3

8.4

28.4

33.7

24.2

100

 

 

 

 

 

 

 

 

 

 

The Study Island exercises helped me comprehend math.

f

6

3

26

32

28

95

33

P

6.3

3.2

27.4

33.7

29.5

100

 

 

 

 

 

 

 

 

 

34

Completing the Blue Ribbon assignments each week increased my confidence in my math skills

f

8

8

29

22

28

95

 

P

8.4

8.4

30.5

23.2

29.5

100

 

 

 

 

 

 

 

 

 

35

Completing the Blue Ribbon assignments each week extended my math knowledge

f

5

9

23

37

21

95

 

P

5.3

9.5

24.2

38.9

22.1

100

 

 

 

 

 

 

 

 

 

36

Completing the Blue Ribbon assignments each week was mentally engaging

f

5

10

27

28

25

95

 

P

5.3

10.5

28.4

29.5

26.3

100

 

           

Table 4.9 details the student responses to the question, " I set goals for myself each week for * choose all that apply." Although 95 students responded to the survey question, Table 4.9 shows a response level of 155.  The reason for this was because many students selected more than one item.  Study Island was the area that appears most frequently at 49%. The remaining responses are evenly distributed.

 

Table 4.9 Self-regulated goal setting

I set goals for myself each week for *choose all that apply

Activity

f

P

Study Island

76

49.0

Playing games on the main page of Graduation Project 1

21

13.5

Playing games on the external links page of Graduation Project 11

18

11.6

Playing games on sites of my choosing

21

13.5

I do not set goals for myself

19

12.3

Total

155

100.0

 

            Although 95 students responded to the survey question, "If you indicated that you do set goals above, please describe in detail how and what sort of goals you set?," Table 4.10 shows a response level of 105.  The reason for this was because many students listed more than one item.  The items were documented by first mentioned to last mentioned. The majority of students either did not set goals or confused developing a work ethic with goal setting. Many of these statements were similar to, I will attempt to complete all my work when it is due.  (Please note: if there was no response from the student the response was marked as none.  There were several instances where the students made a point of noting none.  This is why the none response is included and all those that opted not to respond were added to the total of students that responded with none.)

 

Table 4.10 Goal Setting Statements

If you indicated that you do set goals above, please describe in detail how and what sort of goals you set.

Goal statements

f

P

Do better each time

14

14.4

Do my best every time

6

6.2

none

34

35.1

Obtain Blue Ribbons for all Study Island assignments

7

7.2

Refers to work ethic rather than goal setting

28

28.9

Strive for perfect score every time

2

2.1

To pass

2

2.1

This response is too vague to ascertain

4

4.1

Total

97

100.0

 

 

            Although only 82 out of 95 students responded to the question, "Which of the things used in Graduation Project 11 assisted you the least with being successful in the Benchmark II test?," Table 4.11 shows a response level of 111.  The reason for this was because many students listed more than one item.  The items were documented by first mentioned to last mentioned. Writing a Blog and Games were reported most frequently as the least helpful assignments. (Please note: if there was no response from the student the response was marked as none.  There were instances where the students made a point of noting none.  This is why the none response was included and all those that opted not to respond were added to the total of students that responded with none.)

 

 

Table 4.11 Least helpful Interventions

Which of the things used in Graduation Project 11 assisted you the least with being successful in the Benchmark II test?

Intervention

f

P

10% goal

7

6.3

Blue Ribbon assignments

5

4.5

Choosing skills to focus on

2

1.8

Games

27

24.3

Math Terms

2

1.8

Study Island

9

8.1

Writing a Blog

46

41.4

Math Videos

1

.9

Other

8

7.2

None

4

3.6

Total

111

100.0

 

            Although 95 students responded to the survey question, " Which of the things used in Graduation Project 11 assisted you the most with being successful in the Benchmark II test?," Table 4.12 shows a response level of 122.  The reason for this was because many students listed more than one item.  The items were documented by first mentioned to last mentioned. The Blue Ribbon assignments and Study Island assignments were more frequently mentioned as being the most helpful (Please note: if there was no response from the student the response was marked as none.  There were several instances where the students made a point of noting none.  This was why the none response was included and all those that opted not to respond were added to the total of students that responded with none.)

 

 

 

 

Table 4.12 Most helpful Interventions

Which of the things used in Graduation Project 11 assisted you the most with being successful in the Benchmark II test?

Intervention

f

P

Setting a goal of 10%

8

6.6

Blogging

3

2.5

Blue Ribbons

28

23.0

Choosing what to focus on

5

4.1

Playing games

12

9.8

Math Terms of the day

2

1.6

None

8

6.6

Other

13

10.7

Study Island

41

33.6

Videos

2

1.6

Total

122

100.0

 

            Although 95 students responded to the survey question, " I would recommend the following changes to assist my fellow classmates and me in developing our Math skills?," Table 4.13 shows a response level of 105.  The reason for this was because many students listed more than one item.  The items were documented by first mentioned to last mentioned. To be marked as other there were no two students with similar responses.  Therefore, though other was second only to the none response, it indicates that most students had very individual perspectives on this item. Having more Study Island assignments and Removing the blogs are the two highest consistent responses.   (Please note: if there was no response from the student the response was marked as none.  There were several instances where the students made a point of noting none.  This was why the none response was included and all those that opted not to respond were added to the total of students that responded with none.)

 

 

Table 4.13 Student Recommendations

I would recommend the following changes to assist my fellow classmates and me in developing our Math skills.

Recommendation

f

P

More Challenging games

3

2.9

More Study Island

13

12.4

More Teacher Instruction

5

4.8

None

47

44.8

Other

20

19.0

Remove Blogging

7

6.7

Remove Study Island

4

3.8

Set Own Goals

2

1.9

Uncertain

4

3.8

Total

105

100.0

 

            Tables 4.14, 4. 15 and 4. 16 were related to whether work in other classes influenced the ability of the students to successfully improve their Benchmark testing results. In reviewing Table 4.14, the students were influenced more by their teachers in other classes than by the supports provided in this course.  Approximately 79% of the students indicated that their other courses affected their grades (as indicated in table 4.14). According to Table 4.15 the Math Courses influenced the students success rate the most. A total of 57.9 % of the students indicate that their math courses influenced their grades and knowledge more than the other materials provided through the Graduation Project 11 course (as indicated in table 4.15).   Only 14 out of the 20 students that responded other provided a response to the question, "If you selected other in the above question, which other course affected your skills the most?" The answers were fairly well distributed with Chemistry and Study Island coming in as the top two other courses to influence the results of the Benchmark II test 

 

Table 4.14 Influence of other classes

Has your work in other classes

affected your skills?

 

f

P

Yes

75

78.9

No

20

21.1

Total

95

100

 

Table 4.15 Courses that Influenced test results

If you answered yes to the above question, which course affected your skills the most?

Course

f

P

Math

55

57.9

English

15

15.8

Graduation Project 11

5

5.3

Other

20

21.1

Total

95

100

 

Table 4.16  Other Courses Influences

If you selected other in the above question, which other course affected your skills the most?

Course

f

P

Science

2

2.1

Study Island

3

3.2

Chemistry

4

4.2

Music

2

2.1

None

1

1.1

General Physical Science

1

1.1

History

1

1.1

Total

14

14.7

 

 

            Research Question 2: Which methods of learning are most beneficial—teacher-guided or self-regulated?

            Through Table 4.13 the student recommendations indicated that the vast majority liked the way the program was set.  This indicates teacher-guided instruction in combination with self-regulated learning was ideal to most students.  Although a small percentage, there are those that would prefer more teacher-guided lessons. Table 4.9 results indicated that approximately 12% of the students did not set goals for themselves.  Approximately 88% indicated that they did set goals for themselves though the majority (49%) selected Study Island.  Study Island provides recommendations for achieving a Blue Ribbon and provides a percentage grade for each assignment.  Tables 4.11 and 4.12 indicate that writing a blog, which is generally considered related to self-regulated instruction, did not assist them with developing their math skills. In Table 4.8, 49.5% disagreed with the concept that writing a blog assisted them with developing their math knowledge;  29.5 % were neutral; leaving a mere 21% that felt writing a blog assisted them with developing their math knowledge. The respondents were fairly consistent when asked questions that relate to self-regulated learning as indicated in table 4.8. There is a range of 4.2%-23.1% that shared disagreement with self-regulated learning statements.  A range of 26.3%-35.8%  were neutral, and a range of 44.2%-66.3% were in agreement with self-regulated learning statements.  There was also steady responses regarding the Study Island and Blue Ribbons assignments, which are teacher assigned and designed.  There was a range of disagreement with statements toward Study Island and Blue Ribbon assignments of 9.5%-16.8%.  A range of 15.5%-30.5% who are neutral, and a range of 52.7%-70.5% that are in agreement.  Tables 4.14 and 4.15 indicate that teacher guided instruction impacts student learning.

 

Chapter V

Summary, discussion, conclusions, and recommendations

Summary of the Study

                        This study examined the effectiveness of a variety of implementations provided to the 11th grade students enrolled in the Graduation Project 11 course at Pennsylvania Virtual Charter School (PAVCS). 

            Though there were 259 students originally enrolled in the course, due to a high mortality rate and the fact that we were only examining those students that were in 11th grade, the number of students studied was actually 191.  The students in this study partook of two Benchmark tests provided by a program called Study Island.  Then 95 of those students also partook of a survey that consisted of Likert scale items and open-ended items. This provided a return rate of approximately 50%.

            Descriptive statistics were used to analyze the data from the Benchmark tests and the survey.  Correlations were used but then discarded as they did not yield a significant correlation in any relevant area. The results of the correlation findings are included in Appendix E for reference. Response frequency and correlation data were explored using Statistical Package for Social Sciences (SPSS) software. Pearson product-moment calculations were used to determine that a significant correlation does not exist within this study.  The only significant relationships were to be expected--those that responded positively to Study Island and Blue Ribbon assignment items responded positively to all. 

 

 

Discussion of the Findings

            The students' grades did not significantly improve between the Benchmark I and Benchmark II tests.  The survey indicated that the majority of students were influenced more from their math courses than from the Graduation Project 11 interventions. This study also found that these overall low achieving students have a tendency to associate will power statements with goal setting. This supports Zimmerman and Pons (1986), in their findings that low achieving students were twice as likely to make will power statements as high achieving students were.  Since many students in this study struggled with setting their own goals, this study supports the study conducted by Zimmerman and Pons (1986) that found 91% of the students could be correctly classified (as either high-level or low-level learners) using their self-regulation strategies.

            Low-level learners see ability as a stagnant and immovable trait, but high-level learners see their capacity for learning as something that can change and increase (Malpass et al., 1999). This is supported by some of the recommendation statements of unsuccessful students.  Some indicated that they were not good at math and nothing would ever change that.

            Demographic details were collected through the Study Island program.  These data were not reliable due to the large mortality rate that is clearly overlooked by the Study Island program that was used to collect and analyze the data.

Conclusions

            The results of this study are inconclusive. The mortality rate is too high.  The participation rate is too low.  The lack of reliability and validity of this study make it impossible to draw conclusions.  There were too many other factors that influenced the outcomes of the study.

 

Recommendations for Further Practice and Research

            Based upon the findings and conclusions of the researcher, the following suggestions are presented:

1. Studies should be conducted within a more controlled environment.

2. A virtual school has too many variables to make any strong conclusions.  Research on how to resolve this issue would be truly beneficial.

3. A comparison of Benchmark test scores to Pennsylvania System of School Assessment (PSSA) scores would make an interesting study

4. A study could be conducted that compared the survey results to the Benchmark results of each student.

5. Since there are four Benchmark tests, comparing the first Benchmark test to the fourth test might provide more significant results.

6. Obtaining demographic details in the survey may provide further insights.

7. I recommend that the students no longer write blogs.  Though writing the blogs may cause the students to reflect, most considered it a method of documentation.  So, recommending that the students keep a personal journal for their own records would be more beneficial.  That way those that did glean something from that experience would still have that opportunity and the rest would do better to complete a quiz or something that is more engaging for them.

 

References

 

(2009). Disaggregate. Encarta world English dictionary. Retrieved (2009, October 26) from http://encarta.msn.com/dictionary_/Disaggregated.html

(2009). Goal. Encarta world English dictionary . Retrieved (2009, October 26) from http://encarta.msn.com/dictionary_/goal.html

Common Wealth of Pennsylvania Department of Education. (2008, November 10). Charter annual report. Retrieved from http://www.pde.state.pa.us/charter_schools/lib/charter_schools/PennsylvaniaVirtualCS-2008to2009-CharterAnnualReport.pdf

Fuchs, L.S., Fuchs, D., Prentice, K., Burch, M., Hamlett, C., Owen, R., & Schroeter, K. (2003). Enhancing third-grade students' mathematical problem solving with self-regulated learning strategies. Journal of Educational Psychology, 95(2), 306-315.

Henderson, S., Petrosino, A., Guckenburg, S., Hamilton, S. (2007/December)Measuring how Benchmark Assessments Affect Student Achievement [Electronic Version]. Institute of Educational Sciences. U.S. Department of Education. Retrieved September 29, 2009 8:05PM from the World Wide Web: http://ies.ed.gov/ncee/edlabs/regions/northeast/pdf/REL_2007039.pdf

Kramarski, B., & Zoldan, S. (1008). Using errors as springboards for enhancing mathematical reasoning with three metagocnitive approaches. The Journal of Educational Research, 102(2), 137-151.

Malmivouri, M. (2006). Affect and self-regulation. Educational Studies in Mathematics, 63, 149-164.

Malpass, J.R., O;Neil, H.F., & Hocevar, D. (1999, May/June). Self-regulation, goal orientation, self efficacy, worry, and high stakes math achievement for mathematically gifted school students (FN1, 2). Roeper Review , 21(4), 281-288.

McMillan, J.H. (Ed.). (2008). Educational research fundamentals for the consumer, fifth edition. Boston, MA: Allyn and Bacon

Sacks, L. (2009/March) Galileo and Interim Assessment [Electronic Version]. Office of Strategic Planning, Research, and Evaluation. Retrieved September 29, 2009 8:50PM from the World Wide Web: http://74.125.93.132/search?q=cache:lCBRRmr0mc0J:www.doe.mass.edu/research/reports/0309galileo.doc+Galileo+and+Interim+Assessment+site:edu&cd=2&hl=en&ct=clnk&gl=us&client=firefox-a

Schunk, D. H. (1985). Participation in goal setting: effects on self-efficacy and skills of learning disabled children. The Journal of Special Education, 19(3), 307-317.

Zimmerman, B.J., & Pons, M. M. (1986). Development of a structured interview for assessing student use of self-regulated learning strategies. American Educational Research Journal, 23(4), 614-628.

Click on the links below to access the appropriate Appendix:
 

Appendix A:Insitutional Review Board Form
Appendix B:Principal Permission and Subject Information Consent Form
Appendix C:Survey Instrument
Appendix D:Disaggregate information from Study Island (This information is private )
Appendix E:Correlation Charts and Tables (This information is private )