Why Do We Evaluate?

The post below originally came from work I am doing in a post-graduate class on program evaluation. Current evaluation practices are designed and implemented within school systems that are ill-equipped to truly see the forest from the trees due to ingrained historical practices that date back 150 years. So, my question is: Do institutions of education have the ability and capacity to make the types of changes that digital learning demands in the new era of learning personalization?

Rhonda: You bring up an interesting point. In the evaluation processes that are being discussed in this thread, it is primarily the holders of power who are conducting the evaluation within the school system. This completely cuts out the groups who are the end users, in this case students and by extension their parents. As technology allows for more personalization and customization of learning experiences, the traditional model of schools creating common curriculum for mass delivery is at risk. It is a 20th century, outdated model. Programs like RTI and PBS are designed to provide supports for students struggling to fit within the current parameters of our age and time based system. In the near future (within five years), I believe that there will be a revolutionary shift in the way students are educated due to the disruptive nature of technology. For institutions of education to survive, evaluation should shift from past programs towards future practices. Technology tools have finally caught up with the reality that information was liberated from the school house walls with the widespread adoption of the internet in 1994. Until now schools did not have to adapt because the electronic delivery mediums were not diffused enough in society. The past three years of technological change have totally changed this. What you see now is a tension between large, bureaucratic institutions still trying to hold onto old systems and individuals who can learn at anytime, anyplace, and anywhere. If schools do not evaluate why they still insist on holding students for 13 years in a lock step progression when the tools are available for individualized, self-paced learning experiences, then they are at risk for marginalization and perhaps extinction. The current funding crisis in not going to be solved in the near term. So, what can schools do differently? (This is why I would make the case for an external evaluator on this issue. Schools just don't have the reflective capacity for an evaluation that tackles these tough questions). Here are some other resources on this issue:

Ken Robinson TED Talk:

http://www.TED.com/talks/lang/en/ken_robinson_changing_education_paradigms.html

Cathy Davidson (book)

Now You See It

Tom Vander Ark (book)

Getting Smart - How Digital Learning Is Changing The World

Infographic

I found this info graphic while reading Elisabeth Engum's 'Flipped Classroom Weekly' @PGelisa

It was part of a larger article on flipped instruction by Susan Murphy @suzemuse

Twitter Chats For Education Calendar

I ran into the following calendar quite by accident:

Twitter Chats For Education

#edchat Tuesdays at 7 looks interesting.

What a great way to have a idea about when to follow and participate in a hashtag organized discussion!

Test What Really Matters

I'm (slowly) going through the Ohio Summit 2.5 twitter backchannel and making good on my promise to write about the best of what occurred.

Stan Hefner, the Ohio Schools Superintendent, made the comment that future jobs will require collaboration and innovation. He went on to note that jobs that don't require these will digitized and shipped overseas.

This is the argument that Dan Pink makes about left brain, process oriented work. Pink says that if there is a script, a specific set of rules, or only a single outcome for task, it is a job that can be done by computer or cheaper by foreign workers.

This phenomenon has gone past blue collar jobs and is now steadily attacking white collar (formally safe jobs) as well.

For our educational system, I believe the State Superintendent is correct in his assessment. If he means it then he will advocate for assessment reform that is far more radical than the multiple choice re-design I fear we will get from PARCC and Smarter Balance for the Common Core assessments.

At a very minimum, every student should be required to create a personal, portable digital portfolio of their work as a requirement for graduation. At every grade level there should be a menu of options as to the types of submissions that need to be included. As students progress through their educational career, products can be based on collaborative experiences and focused on authentic problems that need to be solved in the local community. Upon graduating, students will have a rich product that demonstrates what they were able to create with, utilize, and extend the knowledge they gained from their experience in school.

A portfolio mandate would do several things. First, it would divert the fixed, unwavering glare that educational institutions have on directing instruction (the means) towards standardized assessments (the ends). Second, it would force schools to examine how collaboration and multiple pathways to demonstrate understanding of content can be embedded into every lesson. Third, portfolios would require teachers to abandon the notion that a project can only happen once a (quarter, semester, year, whatever). Finally, it would enfranchise learners who have been marginalized by the last twenty years of the "one size fits all" approach that standardization forces.

21st Century skills, at their core, require students to demonstrate how they can apply knowledge in new, unique, and unexpected circumstances. They require flexible, nimble, creative thought. This is NOT what is fostered in the single correct answer environment that has been propagated by standardized testing.

We can not expect this type of learning to be encouraged if the testing mandates continue to promote standardization (the ultimate 20th Century Skill).

Accidental Investigation

The job for this morning was to investigate "two points in time". In other words, how can student growth be measured in non-value added subjects in order to satisfy the student growth measure requirement of the new Ohio Teacher Evaluation System.

I found a couple of other things to share instead.

First is a video link from Learning Matters (a part of PBS) that examines the Mooresville, NC school system's full adoption of technology for learning. It is a 10 minute video that does a decent job examining the benefits and challenges of this approach.

http://www.youtube.com/watch?feature=player_embedded&v=Yt-i--gBa48

The major question that still concerns me is how to you balance the tension between the drive to standardize learning outcomes and the inherent non-standardization of personalized learning enhanced by technology?

If everyone is still expected to complete the same assignment, in the same time frame, but with digital tools, then we should not be surprised when we see issues arise quickly with disengagement, time off task, apathy, etc.

The key to future learning is Authentic Engagement. If you are going to bring digital tools into the classroom, you have to be willing to let students chart their own learning course and demonstrate mastery in a variety of ways (this is what I refer to as the multiple pathway option). If you standardize too tightly in a digital environment, you will have put a lot of money down an educational sinkhole for shiny new tools that students will shun.

The reality is that today, online, you can learn anything, anytime. This is not a new concept to anyone. How are schools going to adapt to this reality? By and large, they haven't, which has lead to a dramatic rise in student apathy. I think believe that students reserve most of their mental effort for work and material that engages them....and these types of activities are not found in the classroom.

If the Common Core assessments focus on specific knowledge, then we are no further along than where we were under NCLB, because teachers will teach to the test in order to "prove" they are doing their job.

This (kind of) leads to the second thing I found (also from Learning Matters). It is a blistering critique of the Common Core by Susan Ohanian. The full text of her thoughts are embedded in a larger discussion of the common core and is linked below:

http://learningmatters.tv/blog/web-series/are-common-core-standards-good-or-bad-for-education/8280/

Here is her website:

http://www.susanohanian.org/

Flip It

Cathy Davidson has been showing up in the twittersphere a lot recently for her work on attention blindness in Now You See It: How the Brain Science of Attention Will Transform the Way We Live, Work, and Learn.

http://www.cathydavidson.com/

One of my takeaways from reading her book is this:

The narrow focus that educators have had on standardized tests over the past twenty years has caused attention blindness in terms of being able to think about multiple pathways that students can take in the classroom. The intense emphasis on students being able to select a single correct response out of four distractors on a high stakes year end assessment has stunted educators abilities to image a classroom where standardized instructional methods (i.e. everyone at the same place at the same time) are not the default norm.

This is why concepts such as differentiation and flipped instruction run into trouble as soon as they are brought up in professional development meetings. Teachers immediately dismiss them because these concepts don't fit into the classroom as it is currently constructed in their psyche. You can probably hear the chatter now...."Well, those are nice strategies, but how can I do that and still prepare my students for the test?".

As the common core and the next generation of assessments come on board, I don't see the emphasis on standardized testing going anywhere. So, how do we break the hold that 'the test' has on our nation's classrooms?

I believe it is going to take courageous teachers who flip their instruction and place standardized, multiple choice tests at the 'floor' and not the 'ceiling' of learning experiences.

Here is what it might look like in my classroom. I would begin the unit with a preview activity of the concepts that students will master by the time the unit draws to a close. It would be nothing more than a bulleted list for awareness. This list would also serve as the test out option on the front end for students who brought prior knowledge into the classroom (Straight knowledge and significance identification).

For the duration of the unit students would have the ability to pick from a menu of activities that are designed to help them learn about the concepts that are new to them. Every item on the menu would have a point value, and students would have to earn a minimum number of points in order for the requirements to be considered complete. The menu would be designed to engage multiple learning styles and interests. For traditional students, textbook readings and associated questions would be an option at the low end of the scale. For students who embrace 21st century learning styles, there would be options to demonstrate mastery utilizing a variety of electronic mediums. The more engaging and authentic the option, the more points the activity would be worth. In theory then it is possible that students could spend the entire unit on one comprehensive project in order to earn the menu points, or students could engage in multiple lower point activities.

Now, the fixed mindset person (Thanks Carol Dweck - http://mindsetonline.com/) has probably already found the hole in the scenario above, as you can't guarantee uniform coverage in this decentralized environment. While this is true, I'll argue all day long that this type of authentic engagement will ensure long term retention with more certainty than a standardized teacher centered delivery model.

The way around the coverage argument lies in how you close out the unit. The concept sheet that began the lesson is now used again in a cooperative activity to detail all of the collective knowledge that students have accumulated over the unit. Gaps can then be filled in by the teacher should they appear. This then turns into the study guide for the summative end of unit assessment. Here is the only time where multiple choice questions are used. The secret is to not instruct with an eye towards the final measure (that would be the ceiling approach), but rather treat the questions like a quality control check. The mindset should be that because of the rich, authentic, engaging experiences the students have been involved with, success on the summative measure should be a forgone conclusion (this is the floor approach).

While I've referenced the above exercise as a summative measure, it can be formative if two conditions are met. First, analyze and disaggregate the data to identify areas that need to be retaught. Second, allow students to complete an alternate measure after the re-teaching to assess growth (this could be as simple as an alternate form of the test, an oral examination, a written response to prompts, etc.) For successful, self-acutalized adults, learning does not stop when a setback occurs; rather, it begins. So it should be the same with our students and the ways in which we design multiple opportunities to demonstrate mastery.

The instructional design concept outlined above is not new. In fact, pieces of it have been floating around in educational circles for years. To embrace it though means to have the courage to set your students free to discover knowledge under your guidance, and not your absolute direction. Instructional design in this manner will allow students to play to their strengths and deeply encode knowledge. Do you have the courage to try?

Note - If you found this post and read all the way to this point, thanks! If you have feedback (good or bad), I'd love to hear it. This blog is a way for me to flesh out ideas that I have about improving brick and mortar education in a time of immense change. Your voice will only help in that process.

Remaking Education?

An article is below that captures the thoughts of the Ohio Superintendent For Public Instruction on college and career readiness.

My major concern is that while on the surface he talks about 21st century learning, students going deeper with the curriculum, improved rigor, etc., his remedy for our educational 'woes' will not measure these 'solution' skills.

In Ohio, not only are we going with PARCC to assess (in a standardized way) the common core, but there is also a push for standardized end of course exams and standardized assessments at two points in time for all courses in order to measure teacher effectiveness.

Isn't this over-reliance on standardized tests and the narrowing of the curriculum what the common core and the focus on 21st Century skills were supposed to get away from?

From my vantage point, it looks like we are heading for a huge spike in standardize tests, not less.

And, if you think that teachers will embrace the types of creative reforms meant to produce flexible, creative, authentic thinkers (the kind that can't be outsourced by a computer if you are a Dan Pink fan) given this new onslaught of testing, I'd think again.

In Ohio current law mandates that 50% of a teacher's evaluation be tied to measures of student growth by the 2013 - 2014 school year. In this economy, if your job is on the line, what do you think is going to happen? More teaching to the test, more top down instruction, less creativity, fewer opportunities for students to demonstrate mastery in unique and creative ways (21st Century), more student disengagement than ever.

At some point this tension between the authentic, immersive, engaged world that students live in outside of school and the authoritarian, standards driven, narrow approach that schools take in the name of proving mastery is going to have to come to a head.

We can not continue to talk about promoting digitally proficient, flexible, creative students while measuring them in ways that do not promote these values.

If traditional brick and mortar institutions continue to keep the current head in the sand approach, competition from electronic providers who understand how to leverage the world of our digitally native students will threaten to overwhelm the traditional system.

A middle of the road solution would be to require a portable electronic portfolio, tied to the common core, that would contain specific requirements (with flexible options) to demonstrate mastery of content at each grade level. By the time students reach 12th grade, this portfolio would serve as a rich senior capstone experience that demonstrates growth over time and authentically measures students strengths. While there would expenses on the front end in terms of planning and implementation, this would allow teachers to embrace the types of authentic, non-linear practices that our students need to thrive in an age where "learn-unlearn-relearn" must be the focus in our hyper-speed global economy.

None of the above can happen if standardization is the only focus.

http://www.dispatch.com/content/stories/editorials/2011/12/15/revamping-education.html