NAEP Results: Less “Bang for Our Buck” (But Plenty of Whimpers)

“Between the idea and the reality . . . falls the shadow.”      —T.S. Eliot

A new report from our friends at the National Assessment of Educational Progress (NAEP), also known as the “Nation’s Report Card,” provides data on student performance in reading and mathematics across multiple grade levels across the country. This latest report shows us how well American twelfth graders performed in 2015 as compared with the last test administration in 2013.

Those were two years of contentious Common Core adoption in many states, or resistance to Common Core and reliance on existing standards in other states—two years of hard work at educational reform and improvement, wherever you lived, whether you were changing course or staying the course; two years of teaching, reaching, explaining, begging, and maybe even bribing students to achieve.

So, what’s the result of our efforts over the past two years? Well, according to NAEP:

In comparison to 2013, the national average mathematics score in 2015 for twelfth-grade students was lower and the average reading score was not significantly different.
In comparison to the first year of the current trendline, 2005, the average mathematics score in 2015 did not significantly differ. In comparison to the initial reading assessment year, 1992, the 2015 average reading score was lower.

Not what we wanted to hear, is it? Either nothing changed, or things got a little worse – hence the quote at the top of the page. Between our aspirations and our actuality, there always seems to be a little shadow—a little gap—a little, “Sorry, not quite.”

Why? What went wrong? Or, more accurately, what’s going wrong, day after day, week after week? Why is that shadow falling between what we’re trying to do and what we’re getting?

MIND THE GAP

Part of the performance gap comes from an implementation gap—the shadow that falls between our plans and the way we put those plans into effect. New standards or textbooks or pieces of whiz-bang software may be brilliant and revolutionary in theory, but if they’re rolled out to schools ineffectively, or haphazardly, or without real buy-in and understanding from teachers. Eventually they wind up on the trash heap and reinforce our feeling that nothing ever works.

Were all of those abandoned programs and initiatives really terrible? Probably not. In fact, most of them were probably fine—maybe even better than fine…in theory. We just didn’t use them properly, or hold onto them long enough to see a result. Anything new requires a little patience, a little persistence. You would never buy a packet of apple seeds on Monday, plant them on Tuesday, and expect a glorious, fruit-laden tree by Friday. But that’s pretty much what we do in our schools, year after year. If the new thing doesn’t work in its first year of implementation, we give up on it and go back to whatever it was we were doing before. Our “flavor of the month” approach to reforms and resources may be one of our problems.

Another problem is that we aim for real thought from our students, but too often settle for mere response. If we’re not aware of that gap, it can cast a lethal shadow over all of our “college and career readiness” initiatives. Here are a few recent examples I came across in my travels:

THE VOCABULARY QUIZ

I visited with a high school teacher several weeks ago—a bright and capable young man who teaches Japanese in an excellent private school. He has been successfully teaching Japanese for a number of years already; his kids get good grades and their parents have been happy. He usually teaches in a fairly traditional style—a lot of lecture, a lot of worksheets, a lot of rote memorization. Pretty standard stuff.

But after a PD session at his school, he decided, as an experiment, to change how he assessed vocabulary. Instead of giving his students a traditional quiz on the words they had been given to learn (here’s the word; choose the correct definition from among four choices), he asked his students to use each word in a couple of sentences. The result was disastrous. They could identify the meaning of the words, but they couldn’t use them. They couldn’t do anything with what they knew.

WHAT’S IN A NAME?

In an elementary classroom at another school, I saw a teacher leading an activity in which students generated lists of nouns, adjectives, and verbs, which the teacher placed up on a large, color-coded chart. The students were asked to create sentences by choosing a couple of the nouns, one of the adjectives, and one of the verbs. When I arrived in the classroom, the students were almost done; most of them were drawing pictures to illustrate their sentences. All the kids looked happy and successful.

But when I leaned over and asked one of the girls which word was the verb, she had no idea…even though it was colored blue on her paper, just as it was colored blue on the chart, under the title, “Verbs.” When I asked her what a verb did a sentence, she didn’t know. When I told her what a verb did in a sentence and then asked her which word in her sentence was doing that, she didn’t know. And she wasn’t the only one who was having this problem. For many of the students, the sentences looked fine. Their sentences were, in fact, correct. They were able to respond successfully to the instructions and complete the activity to the teacher’s satisfaction. But they couldn’t talk about what they were doing, and they seemed not to understand what it is they had done.

SUNK IN CARBON

When I came home from my most recent travels, I saw my sixth grader hard at work on a science assignment. I asked him what he was doing, and he showed me a worksheet about the carbon cycle. The question he had just completed read, “How does deforestation affect the carbon cycle?” His answer was, “Trees are carbon sinks.” He was very happy with his answer, because it was factually correct. He could even show me, in his textbook, where that factual detail lived. But his answer, while true, didn’t respond to the question he had been asked. It took several minutes of (gentle) browbeating and asking “so what?” to get him to connect his fact to the idea of deforestation. He had a lot of facts ready at hand—in his brain and in his notes—but he wasn’t sure what to do with them.
It’s an easy miss on a homework assignment, but it’s exactly the kind of thing we need our teachers to tease out with students. Are our teachers taking the time to help students connect thought to thought, and idea to action, in a way that helps them make the things they’re learning useful?

WHAT DOES THE DATA MEAN?

Every test is a transfer task—you have to take what you learned in the classroom and apply it somewhere else. But no test prep can prepare you for every question or question type you may encounter in the world. You have to be able to come at any challenge with a deep understanding of the relevant content and an ability to be flexible in the way you use it. You have to be ready to improvise at a moment’s notice. You have to be able to think about what you know.
This is why athletes need more than drills. They need scrimmages—practice games—to get the experience of making decisions and using their skills in the crazy, unpredictable, changeable context of a game.

The question for us is: are we deficient in our skills drills, or in our scrimmages?

I don’t believe that a stagnation or slight downturn in NAEP scores means that our teaching is deteriorating, or that a particular class of students isn’t as bright as the class that came before it. We’ve been at the lower levels of Bloom’s Taxonomy—teaching, questioning, and testing at the levels of basic knowledge—and we’re still pretty good at that level. But we’re less effective at getting kids to think about what we’re teaching them so that they can use what they learn confidently and in a variety of new ways.

The more our assessments move away from basic question-and-response—the more they try to present students with authentic thinking and reasoning tasks, the more we are liable to see a shadow fall between what we’ve taught them and what they can do with what they know.

The View from SXSW: Finding Innovation, Optimism, and Passion in Education

“I had been my whole life a bell, and never knew it until at that moment I was lifted and struck.”

Annie Dillard

The South by Southwest Education conference just wrapped up, and I thought I’d take a moment to share some notes and thoughts for anyone who might be interested. Before registering, I hadn’t even been aware that SXSW had an education conference. What did I know?

Education conferences are usually hit-or-miss affairs: some sessions are interesting; some are dull and ordinary. Some addresses bring new information and insight; some rehash things you’ve heard a million times before. This one was no different, although the things that hit, hit hard and rang out loud and clear.

A theme that resonated throughout the event was personalization. It started with Temple Grandin discussing the Autism spectrum and the importance of honoring different minds and the different ways in which those minds see and work in the world. The theme was picked up later in the week by futurist and game designer, Jane McGonigal, who challenged us to see signals of possible futures all around us—and then grab the signals we like and build a new world. She imagined for us a world in which every act of learning we participated in, whether as children or adults, in school or out, earned credits like Bitcoins and contributed to a dynamic, endlessly expanding transcript and résumé viewable by employers and colleagues all around the world. We heard from an author challenging the entire idea of “average” as a way of measuring or describing people—people being far too jagged and variable to be able to be contained or defined by a single value. We saw how the distribution of scores along a traditional bell curve disappeared when students were allowed to work at their own pace and take assessments when they were ready for them, instead of when a teacher assigned them.

Some speakers acknowledged that a measure of standardization and structure still had value and importance to us; some didn’t. All of them wondered: can we de-standardize and de-institutionalize the way we teach and learn, while still holding to some set of goals and standards for knowledge and performance?  If we personalize everything, do we lose all sense of common knowledge, core understandings, shared culture?  Can we still be an “us” if we allow every individual, “I” to design their own pathway and ecosystem of learning?

Obviously, the conference raised more questions than answers—as it should have. But the one answer I did find was that there is an abundance of passion in American education, and an abundance of innovation.  That much was obvious—and it was refreshing and invigorating to see. You know how it is: the day-to-day dramas and traumas of teaching and leading in schools can wear you down and rob you of hope; from the ground-level, it can look as though change is hopeless. But it’s important to remember that ground-level is not the only perspective—and, in this case, it’s not the most useful one.

One of the last sessions I attended was about a group of poor and disadvantaged students from Washington, DC, who had participated, 25 years ago, in an intensive program to help them succeed in school and get into college. Film clips of the students, then and now, were shown (part of a new documentary, called Southeast 67). Some of the students had gone on to college right after high school; many had not. The teacher who ran the program, who now leads efforts on college access at the College Board, told us that at the time, he and his colleagues were saddened by what they perceived to be their failure. All of the support and services they had provided were meant to get those kids into college, and it hadn’t worked. And yet, now, years later, they could see that many of the kids had made it to college—eventually. Some were still in school today. And even if they hadn’t gone to college, all of them had made it, firmly and successfully, into the middle class. Most of them had happy families and children. And all of the children—all of the children—had gone to and graduated from four-year colleges. Many had gone on to graduate school. The investment these adults had made in a group of children had not only changed their lives for the better; it had changed the history and the trajectory of their families. And that wasn’t due to some high-tech innovation or radical restructuring of the school. It was because they had decided, as the president of Franklin and Marshall College put it, that their job was to know those students, love those students, believe in them, invest in them, and never let them go.

Every once in a while it’s really important to be in the room when a story like that gets told. It makes a difference when you can feel the passion and the resolution of the people sharing their story. It makes a difference when you can feel the hope and optimism beaming out into the room. It’s a physical sensation, humming through you the way live music does. They play their music, and the air carries it into your bones—a sympathetic vibration. They ring the bell inside their souls, and the sound waves reach out and ring yours.

Every once in a while it’s nice to be reminded that you, too, are a bell, just waiting to be struck.

Academic Intervention: What Does it Really Mean?

Outside the classroom, the word “intervention” has pretty clear associations. Think of the literal meaning of the word—a coming between—and how it manifests itself in our culture. We all know of instances where people have had to place themselves in someone else’s pathway and make them travel in a different direction, usually because their current pathway is leading to danger or illness (we’re here because we love you, and we need to talk). Sometimes it’s simply behavioral; sometimes it’s medical, requiring the identification of a problem and the prescription of a treatment (you have an addiction; we’re sending you to 28 days of rehab). These associations are so much a part of our culture that movies and TV shows can make quick, easy reference to the word, confident that we’ll all understand what it means. The sit-com, How I Met Your Mother, even made the concept of intervention into a running joke, as the friends intervened with each other to stop a whole range of annoying behaviors (to the point where they had to stage an intervention to step each other from having interventions).

In our world of education, the word is used pretty frequently. But when we implement an academic intervention, it’s often just a repetition of what was already taught. We send kids to summer school to learn the same content they learned from September to June, often taught by the same teachers using the same materials. Or we send them to a separate room during the school day in a pullout program, to learn the same material with a different teacher. What we don’t do—nearly enough—is stand in a child’s pathway and help him take a different direction. Instead, we walk him back to the beginning of the journey and ask him to retrace the steps he’s already taken. It’s not a new show; it’s just a re-run.

I’m not saying repetition is a bad thing. Some students definitely benefit from repeated exposure to material. For some students, that second time through is when things finally stick. But what about the students who need more than a re-play? What about the most challenged students, for whom the material or the instructional approach simply don’t click—don’t make sense—don’t help?  These students don’t need to have the material served up again, faster and louder. They need a real intervention; they need someone to stop them from going along the road of error, and re-direct them on the pathway of learning so that they can reach success. Stop doing it this way; try doing it that way instead. Stop thinking about it this way; think about it that way instead.

Being Diagnostic: Stop Doing That

There’s an old joke:  A guy walks into a doctor’s office, swings his arm around, and says, “Hey Doc, it hurts when I do this.” The doctor says, “Well, stop doing that.”

This is the job of the doctor: to figure out what ails you and then make it better. The first part can be very tricky (as fan’s of the old TV show, “House,” may recall).  When you walk into a check-up feeling healthy, a good doctor will poke and prod to make sure you really are healthy. When you walk into an appointment because something is wrong (perhaps with your arm), the doctor will poke and prod to figure out what the problem is.  You may think you know (“It’s carpal tunnel, Doc. I know it!”), or you may just be in pain. The good doctor takes nothing at face value—he assesses the situation to figure out what the problem is. Sometimes that means running tests; sometimes that means asking a lot of questions beyond the test.

In academic intervention programs, we often talk about being “diagnostic and prescriptive,” but our diagnostic process is often woefully inadequate. We certainly have our tests. We have instruments designed to find out what our students know and can do. If students miss a question or two, we can identify areas of weakness or concern. But our test instruments, whether standardized or teacher-made, rarely go deeper than that. They barely tell us what; they almost never tell us why.

Imagine a doctor who can establish that a patient’s arm hurts, but has no tools or skills to tell him why. What use is he to the patient?  “Well, sir, it looks like your arm hurts.” “Thanks, Doc. That much, I already knew.” Imagine the doctor who reads the results of a blood test but doesn’t bother to talk to the patient about her family history, her diet, her lifestyle, to find out what lies behind the data. We would all probably agree this was not a great doctor.

I mentioned during a recent presentation that teacher questioning shouldn’t be aimed solely at the correct answer, but should also drive students towards revealing mistakes and misconceptions. When I said that, I got some horrified looks in response. But it’s true; the correct answer is the least interesting piece of information in the room. We already know what the correct answer is. Figuring out who else knows it is lovely, and important, but it’s simply a confirmation that everything’s great. What if everything isn’t great?  What if someone is sitting there, hopelessly confused—the patient who is in pain but refuses to acknowledge it?  What we need to do is poke and prod until somebody says, “Ouch!” Until that happens, we can’t find out where it hurts—and if we don’t know where it hurts, we can’t help.

This is especially tricky in the classroom, because in many cases, students can identify a correct answer or solve a problem without having any idea why that answer is correct or why they did whatever they did to solve the problem. They follow a process that they don’t really understand, and it seems to work. Or they repeat something their neighbor said. These tactics leave them extremely vulnerable and open to future error—and lead to bewildered teachers saying, “But he knew it when I taught it.”

If we don’t take time to ask, “Why?” or “How?” or “How did you know?” we miss our chance to peek inside the child’s head and see what’s going on in there. And if we don’t take time to structure our questions and examples correctly, we miss our chance to diagnose the problems we find there.

Elizabeth Green, in her book, Building a Better Teacher, provides fascinating examples of teachers who know how to ask probing, diagnostic questions that reveal misconceptions and mistakes. It’s a time-consuming practice, to be sure. It means resisting the easy path of Xeroxing worksheets from a textbook or downloading activities from the Internet. It means really thinking through the thought-process behind the skill or concept we’ve taught—how it works, why it works, and how it can go wrong. And that’s tricky, because we’re experts at what we teach, and we may never really think about what we’re doing. The skills are automatic and the concepts are deeply engrained. We take them for granted, and that makes it easy to teach them as if they’re self-evident. If they’re obvious, all you need are some confirming questions to make sure everyone “got it.” 

We need to slow our brains down and think about what those brains are really doing when we find a lowest common denominator, or interpret a poetic image. We need to see the material the way the child sees it, so that we can catch and make sense of the errors they’re making. Maybe not all the time—maybe the way we teach and test and question works perfectly well for 60% of our kids—maybe even 75%. But for those students who get lost in the weeds or the woods—the students who get left behind and end up wandering down a dark and confusing pathway—we need to be ready to intervene.

 

 

Taming the Chaos Within

(originally published at www.catapultlearning.com) 

“How can you make a world for people to live in until you’ve first put order in yourself?”

Thornton Wilder, The Skin of our Teeth

 

Early in my teaching career, my headmaster called a faculty meeting and had us watch a documentary about learning disabilities. Most of the students in our small school struggled with a disability of some kind, from mild dyslexia to almost total aphasia. Some of their parents were caring and concerned; many were frustrated and exhausted. The video, called “How Difficult can This Be?” showed us a workshop during which a facilitator conducted a variety of activities to make participants feel as though they had all sorts of learning challenges. He gave them mangled text to read or fuzzy pictures to look at and then barked at them to “look harder” at things they could not possibly see, no matter how “hard” they looked (whatever that meant). 

 

Even without participating directly in the workshop, we learned a great deal about what it felt like to be our students—students who had spent a lifetime being told to sit still when they were already sitting as still as they could. How heartbreaking it had to be, to be yelled at for not being able to control a twitching hand or a tapping foot. How enraging it had to be, especially by the teen years, to be scolded for not reading things that were physically unreadable. No wonder our students had been in and out of every school in town.

 

To many teachers, the idea of “differentiated instruction” is a ridiculous, pie-in-the-sky idea dreamed up by out-of-touch authors or professors who don’t understand how difficult it is to plan and execute a single lesson, much less several options per class. But to many students, differentiated instruction is the difference between success and failure in school. For many students, instruction that isn’t adapted to meet their needs is instruction to which they simply have no access.

 

Identification of learning disabilities and a well-implemented IEP or 504 plan can make all the difference for these students. But what about students whose problems do not fall into the establish categories of problems that will lead to legally-mandated accommodations? What about students for whom years of poverty, neglect, or trauma have so stressed the neural circuitry that they have not developed levels of executive function at the same pace as their peers?  When they don’t have a medical label to explain away their problems, they simply get labeled Bad and are punished rather than helped. And yet, their experience of school may be no different that the experience of students with identifiable disabilities: it doesn’t work, it doesn’t fit, they don’t know why, and they get yelled at for it.

 

This is tragic for all sorts of reasons—not least of which is the fact that, in the words of author, Eric Jensen, “Brains can change.” Neuroplasticity, the ability of a brain to adapt and change over time, even beyond childhood, means that a student’s past doesn’t have to be his future—that with careful and planful work on the part of the school, even a student with severe behavior issues can learn to control himself and succeed in the classroom.

 

But what does “careful and planful” mean for such students? It doesn’t mean medicating them into a stupor, and it certainly doesn’t mean threatening them with suspension if they take a wrong step. For students who can’t control their impulses or their aggression, it means providing a safe and orderly environment where they can actually, finally, learn to control their impulses and their aggression—where they can regularly practice a menu of self-control strategies and demonstrate increasing levels of self-control. At Philadelphia’s Anthony Wayne Academy, one of our SESI schools, I have seen how even the most behaviorally-challenged students can take control of their behavior and earn increasing levels of autonomy and authority. I’ve seen students who were once sent away from their schools becoming leaders: running meetings, modeling correct behavior, and acting as role models for younger or newer students. They learn how to tame the chaos within themselves, whatever the cause of that chaos may have been, and gain enough self-control to return to their schools, earn their diplomas, and pursue their dreams.

 

That shouldn’t have to be seen as a miracle, or even a gift. It’s simply what those students need. Yes, they need Algebra. Sure, they need World history. But if they can’t tame that inner chaos and take control of their lives, what meaning or use can Algebra or World History ever have?

 

No student in our country should be denied the opportunity to succeed simply because he doesn’t fit into the box we call “school.”  The school should accommodate itself to meet the needs of students, not the other way around. I believed this, way back in the 20^th century, when it felt a little bit radical and strange. In this century, it should be a commonplace. Customization and personalization are simply what we expect—and demand—in most areas of our lives. In fact, I can’t think of any other area of life where we still think it’s acceptable to take a random group of 20 or 30 people (who share nothing but a birth year) and give them all exactly the same thing, delivered exactly the same way, regardless of what they might need or want.

 

Our democracy is built on the assumption that people can govern themselves and manage their own affairs—that we are not forever children, searching for a strong parent to obey. But not everyone is ready and able to take control of their lives—even at age 18. And a person who is unable to manage himself is a person who will eventually be managed by someone else—often the police or some other mechanism of the state.  That is not exactly a gateway to the “pursuit of happiness” that was meant to be our birthright as Americans.

 

We are supposed to be giving our students the tools they need to succeed in life beyond the schoolroom.  But if we can’t see the world through their eyes, how can we possibly know what tools they need?

 

 

 

 

Connecting the Dots: The Essential Relationship between a School’s Mission Statement and Its Day-to-Day

(originally published at www.catapultlearning.com)

At one of Catapult Learning’s recent regional leadership conferences, I spent some time with principals and teacher-leaders from our partner schools. We talked about the importance of connecting the dots from the school’s vision and mission to its values, and then all the way down to the daily tasks that each member of the school family engages in. It’s surprising how disconnected the mission and the day-to-day can be from each other. We write beautiful mission statements and hang signs and banners up on the walls, and somehow we expect the pretty words to exert a kind of magic over us, pulling us into their orbit. Sadly, it takes a lot more work than that.

The “big picture” is made up of a thousand connected details. Little children know this, but it’s an easy thing for adults to forget. Our grown-up connect-the-dots puzzles are less obvious, and they aren’t laid out for us to solve, step by step. We choose our own dots and often don’t know what picture we’re forming when we start connecting them. We may think we’re aiming at our mission or vision statement, but our day-to-day routines may add up to a very different picture.

So, during our workshop, I asked the principals and teacher-leaders to start with the vision and mission of the school and work their way down, to see if they could connect the lofty words with the more mundane tasks that make up our days.

One of our partner schools had, as part of its mission statement, the growth of students into responsible citizens. It was a lovely sentiment, but when I asked the school team what values the adults in the school held in order to make that mission real, they were a little stymied by the question. It’s easy to say, “We believe citizenship is important,” but what does that actually mean, in practice?

At first, there was simply repetition of the words from the mission statement, but eventually, the team came up with this statement of values: “School leadership and staff respect and honor the student voice and promote student autonomy.” That’s a clear statement of a value that can, if widely held, help students understand what democratic citizenship means.

The next step is even more crucial: day to day, what does it actually mean to hold that value? What does it mean for the English teacher? What does it mean for the gym teacher? What does it mean for the bus driver? If we say we respect the student voice and want to give students real autonomy, how does each adult, in his or her own particular role, make that come to life?

One team member said, “Well, we have a student government.” I asked if the student government got to make decisions that affected the way the school was run, or if it just concerned itself with the theme of the spring dance. The question was met with some uncomfortable laughter, but it mattered. How can you promote democratic citizenship if students can’t actually cast a vote that means something?

I pushed the idea a little further: what about the cafeteria staff? Are they involved in these core values? Do students have a say in what gets served at lunch? There was more rueful laughter, but no one responded. “Are we afraid that they’d make terrible choices?” I asked, “That you’d have pizza and doughnuts every day?” People nodded and laughed. “But isn’t it our job to help them become informed voters?” I asked. “Isn’t that an important part of responsible citizenship?” Thomas Jefferson was pretty clear that a democracy required an educated and informed citizenry. So why not provide opportunities for students to learn about healthy diets—not just abstractly in their health class, but as part of their role in helping plan the lunch menu? Wouldn’t that be one real expression of how the school embodies its core value and promotes its mission?

Every aspect of a school’s mission statement should have a corresponding value that requires particular behaviors and real commitment on the part of the school staff. Every one of those core values can be expressed, somehow, in the particular day-to-day tasks of each adult working in the school—even the bus driver; even the janitor. Connecting those dots from the ground floor all the way up to the clouds is rigorous and time-consuming work, but it’s worthy and important work. It’s work that pays off, in the long run.

It’s very difficult to manage a mission or a vision. They’re abstract and amorphous and often dreamy. At the same time, it’s easy to get overwhelmed by the day-to-day needs of the school and feel like you’re on a treadmill of drudgery. But if you’ve taken the time to ensure that the day-to-day tasks connect, in real and meaningful ways, to the mission and purpose of the school, then you won’t have to worry about that mission anymore, and you won’t have to worry that the details are trivial. Make the small details work, and the big picture will take care of itself.