Summary

Delaware began testing online in 2010, acting on a recommendation to support districts with more flexible and better data than was possible with the prior system built to suit a high-stakes testing model. The Delaware Comprehensive Assessment System (DCAS) features summative exams and formative exams that allow for  broad testing windows that allow districts to “test when they need to test”.  With (1) a centralized statewide network that provides Internet connectivity to each school and (2) a statewide pupil accounting system used by every district already in place, the Delaware Department of Education (DDOE) led a field test of the DCAS first online exams that was planned and completed over a very short five month period.  The rapid test was successful because the infrastructure was in place first, field test data was “not counted” (i.e. non- high-stakes), and because the state prioritized highly inclusive communication during the planning and testing process.  Statewide initiatives that call for timely and rich data have been made possible since the full online launch of the DCAS assessment and reporting system during the 2010-2011 school year.

Last modified: 2015-07-06 22:31:04

History and Background

Delaware is a small state of only three counties that encompass 19 school districts (16 traditional districts and 3 county-wide vocational/technical districts) and 22 charter schools.  Approximately 10,000 students are tested at each grade level in 210 schools statewide (105 elementary, 34 middle, 33 high, and 38 special schools).  Here is the interactive map  of Delaware school districts and schools.

The move to adopt and fully implement Common Core State Standards (CCSS)-aligned assessments in 2014 builds on Delaware’s nearly twenty-year history of planning toward better information sharing across the state.  Until 2010, the high-stakes Delaware Student Testing Program (DSTP) was a paper-and-pencil based statewide accountability assessment system that returned summative data annually, typically several months after students completed the exams.  Faced with DSTP testing contract expiration after the 2009-2010 school year, the state formed a taskforce of teachers, administrators, lawmakers, business leaders and Department of Education (DDOE) personnel to conceptualize a new assessment system. The Assessment Task Force identified two priorities for the next iteration of tests: immediate scoring, and the ability to track data over time.  These priorities led to the group’s ultimate recommendation for a shift to online assessment.  DDOE subsequently issued an RFP for a test provider who would work collaboratively to develop, implement, and service the new online, web-based delivery and scoring system for the Delaware Comprehensive Assessment System (DCAS) in all four tested content areas.

Delaware wanted to emphasize testing flexibility both in how the test is administered, and within the assessment. The DCAS provides multiple opportunities for students to demonstrate proficiency and provides academic achievement information to students and parents, including a measure of fall-to-spring and year-to-year individual student growth.  Students complete tests on computers or laptops that have a platform-independent secure testing browser installed.  All questions are multiple choice, with no extended response options.  Math and ELA exams are adaptive, with the potential of up to 20 percent of questions that are off-grade by up to two grade levels.

Currently, Delaware has a law requiring all state-mandated and high-stakes assessments – including summative annual assessments, End of Course (EOC) exams, and/or high school exit exams – to be administered digitally.

Last modified: 2014-09-11 16:41:47

 

Milestones to Online Assessment


composite timeline image for Delaware

Before statewide launch

  • 1990’s-2000’s – Statewide installation of school connectivity (initially, installation of T1 lines to all schools) and on-boarding districts into pupil accounting system.
  • 2008 through 2010 – Migration of schools from T1 lines to the current minimum standard of 10 Mbps (some schools have up to 100 Mbps) for all but 36 schools.
  • Summer 2009 – RFP for online assessment vendor issued.
  • Dec 2009 – Online assessment contract awarded to American Institutes for Research (AIR).
  • Jan 2010 – Official start of work to move assessment online.
  • Winter and Spring 2010 – Monthly statewide stakeholder meetings to prepare for field testing.
  • April 2010 – Conducted a non-high-stakes field test of online assessments immediately following the final paper-and-pencil administration of high-stakes DSTP exams.
  • Summer 2010 – Administration and Legislature support for purchase of 6,088 laptops, 220 laptop carts, and external mice to supplement existing hardware for testing.  Migration of 36 schools with T1 lines to the current minimum standard of 10 Mbps.

Year of statewide launch: 2010-2011

  • 2010-2011 school year – Initial launch of online assessments statewide with fall, winter and spring testing windows, beginning with the following tests, given exclusively online:
    • EOC: English II, Algebra I, Integrated Math I, Biology, US History
      Mathematics grades 3-10
      Reading grades 3-10
      Science grades 5, 8, and 10 (spring only)
      Social studies grades 4 & 7 (spring only)
  • July 2011 – Teachers use DCAS data for the Taking Action with Data (TADa) Framework in conjunction with the Data Coach Project.  Funded with Race To The Top dollars, these initiatives provide job-embedded support through teacher teams that examine student assessment data. All district schools created and implemented 90 minutes of weekly collaborative planning time for every core content teacher in grades three through twelve, scheduled either in one 90-minute block of time or in two separate 45-minute time blocks.

 After full launch

  • 2011-2012 school year – DCAS added online tests for Algebra II and Integrated Math III
  • 2012-2013 school year – All DCAS assessments are mandatory with the exception of serveral EOC exams.  Tests are administered online during fall (Aug-October) and spring (Jan – May) assessment windows.
  • 2014-2015 school year –  Scheduled to implementation of Smarter Balanced assessments

Leaders in Michigan are completing preparations for online assessments to deploy statewide in 2014 in multiple areas simultaneously, whereas other states have proceeded more sequentially.

1490260049

Last modified: 2014-02-04 12:34:09

Readying Users

Delaware’s rapid move to online assessment was made possible by an existing infrastructure comprised of (1) a centralized statewide network that provides Internet connectivity to each school (with few exceptions) and (2) a statewide pupil accounting system that tracks accommodations, unique student IDs, and classroom rosters used by every district already in place.  Additionally, the annual school technology survey revealed that schools had enough computers to conduct the field test on short notice, even though many computers were reallocated away from instructional uses so they could be made available for the initial round of online assessments.

With the technical elements for a successful field test largely in place, the DDOE turned its attention toward generating buy-in among stakeholders statewide.  The DDOE recognized that online assessment affects nearly “everyone”, so they deliberately involved as many stakeholders as possible starting very early in the field testing process.  Curriculum directors, technology directors, instructional technology specialists, test coordinators, principals, teachers, team leaders, superintendents, assistant principals, assessment directors, technology staff and others were invited to ask questions, make suggestions, and devise strategies to assure field testing was a successful learning experience. DDOE utilized existing communication channels such as regular meetings and listservs, and added four countywide stakeholder meetings, each with 200-400 people voluntarily in attendance, to expand the discussion.  Ten readiness trainings were held in February 2010 to prepare for online field testing around the state.  These efforts assured that “When [DDOE] knew it, [educators] knew it”.

A significant factor in Delaware’s rapid transition to online assessment readiness was DDOE’s emphasis on the non-high-stakes nature of the field test.  Rather, early conversations focused on technology capacity and hardware issues, particularly the logistics of setting up schools for online assessment.  DDOE responded by rolling out checklists for before, during and after online assessment that helped address questions and establish best practices. From the very top, the governor’s office, state legislature, and educators actively worked together on the transition: “Any of these three groups can stop these efforts in their tracks…We have been able to enjoy discussion, promotion, encouragement, fiscal resources, people resources–we’ve just had a commitment from all those levels to get it done” (DOE official, cited in Schaffahuser, 2011).

“Just as ‘we didn’t know what we didn’t know’ when we first started, we also knew that was also true about the education community at large. We had to go beyond our normal top-down communication channels and reach out to the masses: assessment coordinators, curriculum directors, IT staff, principals, and teachers to name a few.” – Dr. Wayne Hartschuh Executive Director, Delaware Center for Educational Technology

1490260049

In Michigan, TRIG provides professional development for classroom technology readiness, as well as resources for assessment and curriculum.

1490260049

Last modified: 2014-09-26 16:45:04

What Districts Did

Prior to DCAS field testing in 2010, districts and schools were using technology for instruction, but administering paper-and-pencil assessments.  The broader effort to better align teaching and testing modalities across the state and offer schools more freedom to set testing schedules locally involved more than simply converting paper and pencil tests to a computer. Assessment no longer meant entire buildings would shut down for several days while all students inside completed the exams.  Rather, Delaware’s broad assessment windows, including a 10-15 week window during the field test, enabled classes of students to cycle through testing on a timeframe that work best for their instructional pacing, while other teachers and students in the school can proceeded with “business as usual”.

District technology coordinators were instrumental for making the vision for statewide online assessment a local reality. To prepare for the field test’s first round of online exams, each district or charter school identified a DCAS District Liaison who coordinated: item review for all content areas, vendor-conducted focus groups for scoring and reporting, DCAS School Readiness Teams that prepared schools for field testing, and DCAS test administration certification in their local sites.  The District Liaison oversaw four areas of school readiness for field testing DCAS online assessments:

  • Technology Readiness – Including network, bandwidth and computers readiness.  Prior to field testing, DDOE also worked with schools to download the appropriate testing browser, enter student data, finalize classroom rosters, and transfer other files to prepare for the launch.
  • Teacher/Staff Assessment Readiness – Including training of district technology coordinators, as well as training of school and district test administrators. The Test Administrator certification includes a 1-2 hour training video covering topics addressed in the Test Administration Manuals.
  • Student Readiness – All students viewed a PowerPoint presentation that introduced the DCAS training and field test.  They also completed both a computer skills assessment and a training test (March 2010) before confirming student IDs and taking their first “live” DCAS online exam.  Preparations were also made for students with special needs and in need of accommodations to participate alternatively in field testing.
  • Scheduling – Districts developed careful schedules of students, labs, rooms, and exams so that all assessment could take place within the approved window.

D. Patches Hill, Technology Systems Manager, and Michael Lingenfelter, Test Coordinator for the Indian River School District in Selbyville, DE, reflected on the past few years of preparing for and launching DCAS online. Indian River School District is comprised of 14 buildings serving 10,000 students across 360 square miles.  They recall initially feeling limited by their district’s hardware and infrastructure going into the field test in 2010.  One ongoing priority has been to bolster their computer inventory.  Even in its fourth year, online assessment still has displaced some instruction because there aren’t enough laptops with particular features to meet all demand scenarios.  For example, the base model laptops issued by the state after the 2010 field test were originally expected to be used for online assessment. In reality, the laptops’ short battery life limits how long they can be used continuously, as they are often configured for testing.  One way that Hill and Lingenfelter and Indian River have adapted since the field test is by making any and all hardware available for better alignment of instruction and assessment across the district.  One of Indian River’s larger undertakings was establishing Wi-Fi in every school to assure secure, reliable, redundant connections.  Currently, all but two buildings are wireless.

Infrastructure upgrades at Indian River have supported educators in their work in well-established professional learning communities (PLCs).  Teachers have been participating in professional learning communities (PLCs) designed to increase communication, cooperation and problem solving for the past 11 years, well before online assessment began.  Teachers and leaders on PLCs create an authentic need for data from online assessments through data-focused meetings involving educators sharing strategies and reflecting on practice.  PLCs also provide a structure for decentralized decision making about the timing and configurations for online assessment that work best for the local curriculum scope and sequence.  Despite the considerable flexibility offered by broad testing windows, most schools continue to follow the more traditional approach of holding shorter testing windows comprised of a few days during which nearly all students are tested.

Looking ahead, four of the schools at Indian River SD (2 high schools, 1 middle school and 1 elementary school) are participating in the spring 2013 Smarter Balanced pilot test.  This pilot functions somewhat like the 2010 DCAS online field test, in that many new tools and question types are being rolled out to a broad range of participants, feedback is being collected via survey, and the test is non high-stakes.  Teachers and students at Indian River are gaining practice, comfort and familiarity with upcoming components of the new online assessments through the pilot; benefits shared with educators at Indian River who also participated in the DCAS field test three years ago.

Do you have enough computers for online testing?

Step 1:  At your school, find out how many total classes and students in each tested grade that will take the field test.

Step 2: At your school, how many computer lab(s) are available for testing?

Step 3: At your school, how many class periods per day will be used for testing?

Step 4: Multiply Steps 2 and 3.

Step 5: Divide the total number in Step 1.

1490260049

In Michigan, 5 consortia work with the districts in their areas to ensure equitable statewide engagement in TRIG initiatives.

1490260049

Last modified: 2014-09-26 16:45:47

 

Test Features

Delaware is strongly committed to streamlining the end user experience.  DCAS hosts all state mandated tests through a comprehensive online portal.  Students and teachers log into a platform-independent system hosted on a Firefox interface with one username, password and session ID that gives them assess to the test they are taking, as well as to digital instructional content[1]e.g. Discovery Education.

The DCAS online has two test types:

  1. Combined Summative/Formative assessments that can go off grade for up to 10 questions for math and reading (spring testing window).  Summative assessments only for science and social studies.
  2. Summative End-of-Course (EOC) Assessments for high school in English language arts, mathematics, science, and social studies.

DCAS features adaptive formative assessments in math and reading in which 40 questions are on grade and up to 40 questions can be up to two grades above or below.  Online delivery makes adaptive testing possible, providing teachers with faster, higher quality data for instruction and real-time growth metrics.  Delaware was the first state in the country that has a USDOE-approved growth model for measuring “live” student growth during the school year.

Click here for an overview of Michigan’s 2013-2014 statewide assessments.

1490260049

Unlike the earlier DSTP’s spring-only testing schedule, schools and districts can administer any of the three types of online DCAS assessments during approved broad testing windows throughout the year. Testing windows provide flexibility and autonomy in exam scheduling, as well as opportunities for system maintenance and upgrades.  The 2012-13 school year saw an expansion of testing windows; the spring tests are open from January to May 30.

In addition to hosting tests, DCAS is comprised of a number of applications.  The DCAS Technical Specifications manual offers a comprehensive list of DCAS applications, some of which are briefly summarized here.  Access to these systems are dependent on DCAS user role.

  • Test Information Distribution Engine (TIDE). Serves as the hub for managing system users; permits authorized users such as test administrators, principals, superintendents, and teachers to view student information (i.e., test settings, accommodations).
  • Practice and Training Sites. Allows both testing administrators (TAs) and students to practice monitoring and participating in testing sessions with the online system and become familiar with tools and features.  Anyone can log in as a guest and access practice exams.
  • Test Administration Sites. Permits authorized TAs to create and administer tests, provides access to the Student Testing Site, and permits data and score entry to authorized users. Students log in using a standardized student ID (established by DDOE as part of centralizing their pupil management system prior to field testing).
  • Online Reporting System. Permits authorized users to view student score reports and track student participation.
  • Learning Point Navigator. Offers curricular support resources and enables teachers to create assignments that address standards. Students are offered immediate access to raw scores on assignments, and teachers can view student and class reports.

Starting the 2013-2014 school year, mobile devices can be used for testing.  These devices are to include tablets and iPads.

 

 

Last modified: 2014-09-26 16:48:25

Looking Ahead

Delaware’s readiness for online assessment builds on a history of infrastructure and educator initiatives that have taken place statewide over the past 15 years. Teachers and students are familiar with Smarter Balanced’s adaptive online assessment format from their experience with DCAS.  State and district officials have noticed other states beginning to ask questions that Delaware was asking three years ago as they were getting started. “Been there, done that,” said one DDOE official.  DE is rolling out an updated version of its secure browser for online student testing at the same time other states are starting from scratch. Having the same test developer as Smarter Balanced, DOE officials agree, puts them “at a big advantage;” however it was the rigorous field testing with deliberate and thorough communication that prepared them most for the transition.

Last modified: 2013-07-11 20:19:33