“Our data is showing that students are growing at three times the rate of the national norm—so they're getting three years of growth for every year—on standardized tests because rather than do what most curricula do, which is start at simple text, we're able to start at complex text and really focus in on that.”
This interview is part of our Spotlight Series, learning more about the work of education innovators, practitioners, and researchers. The interview has been edited for length and clarity.
Matt Pasternack: Can you start off by telling me what Riveting Results does?
Arthur Unobskey: What Riveting Results does is tackle what for high school teachers is an incredibly challenging task. Nationally, 69% of entering ninth graders read below grade level. And so in a typical English class, which is typically heterogeneous, a teacher looks out at the classroom and sees more than two thirds of their kids below grade level and then a third at or above grade level. So they've got to teach all these kids, and these kids can range from reading at third grade level all the way up to 12th grade level, and they're all in that same ninth grade English class. And you may have 25 to 30 of those kids. The standards across the country expect you to teach grade level text to them, complex text.
So, how do you make it possible for that to happen? First of all, you need to understand why so many kids are stuck at fifth grade reading level. It's because of what happens between fifth and sixth grade, in which sentences change from simple sentences to complex sentences, and they get longer. The foundation of their skills for learning to read in early elementary are based on phonics, phonemic awareness, sight words, and then, finally, fluency. That fluency breaks down when they start getting into sixth grade and they start tripping over these longer sentences with clauses—complex sentences—and they get stuck and what they do is they start cheating. They start to look for the simple sentences in sixth, seventh, and eighth grade to help them understand with context what those longer sentences are, and that kind of works. You see kids testing out, maybe they're sixth grade level or seventh grade level, but they've never learned how to read complex text. And so by the time they get to ninth grade, they have no facility with it, and those simple sentences have decreased significantly in number, so they cannot read what is put in front of them. And that means they get disengaged and bored, whether it's science, history, even math, but particularly in English class, because they can't read the kinds of sophisticated literature that adolescents want to read. So the teacher’s stuck.
So what we did is took that big blocker, which is fluency, and then three other high-leverage literacy activities and put them into software. So the teacher can take a really engaging, challenging, rigorous novel and teach the whole book to the whole class at the same time. They're all reading the same book but have activities that she can personalize. Different kids are doing slightly different things, practicing the same skills, but at slightly different levels so that they can all participate in really dynamic discussions and partner activities that the teacher leads and the computer software helps guide students. All the work goes through that software, so what happens is that students are spending a hundred percent of their time reading complex text. So even if you're at a fourth grade level, the exciting thing about reading is, it's not like math, you can actually skip levels, because if you can read something fluently—our first activity allows kids to practice their fluency and develop it—you can get to the point where you can kind of understand a ninth grade level text very quickly, within even a matter of minutes, but really become comfortable with complex text within a matter of a few months. And then if you do the other activities, you can really understand complex text, the whole book worth, if you support that with other activities. So our data is showing that students are growing at three times the rate of the national norm—so they're getting three years of growth for every year—on standardized tests because rather than do what most curricula do, which is start at simple text, we're able to start at complex text and really focus in on that.
Matt Pasternack: Are kids physically—I'm trying to visualize the classroom—is it a one-to-one computing environment where they're getting personalized content on their screen? Or are they holding a physical book and the computer is more just kind of guiding the teacher in terms of lesson prep?
Arthur Unobskey: So it's a little bit of both. The structure is that we have taken books that we've tested with thousands of kids that are incredibly engaging or full of complex text and have beautiful stories—
Matt Pasternack: What are some examples?
Arthur Unobskey: So the Autobiography of Langston Hughes, Homegoing, which is a book by Yaa Gyasi that came out in 2016, books that are really engaging but have really complex narratives and stories. And what we've done is we've broken these books up into modules so that there are typically around 15 for each book, and each of those 15 is like a short story; it's typically a chapter but not always. And in that section or chapter, what we do is we laser focus on a particular passage. So students—they have the hard copy of the book—they read the book together. First of all, there's no homework in the program because that is one of the things that makes inequities occur, so everything's taking place in the class. They either read it to themselves or the teacher reads it out loud, that chapter. Then the teacher has assigned them a passage that's particularly important—the teacher has assigned the whole class that passage—to practice reading out loud to themselves. They have headphones on and the software guides them through so that they can prepare to make a recording.
And students who maybe start at the fourth grade level might start with what we call flow, which is the first level of adolescent fluency, but again, because adolescent fluency requires a softer touch than what they did with simple text, they have to actually learn how to impart emotion as they read, learn how to slow down and speed up. So there's different levels of this. So your kid that's reading at fourth or fifth grade level may start with flow and progress up, whereas a kid that's reading on grade level or higher might start with emotion and read a longer passage right off the bat. But they practice repeatedly, over like a five to 10 to 15 minute period, reading this out loud.
Then they're ready, prompted by the software, to make a recording. They make a recording; they send it in. We have remote scorers who then score all these fluency recordings, send it back, the teacher sees this on a data display and can make a decision. The next day, do the students need to do more flow recordings or can they progress to emotion? And maybe they do that in a new section, but they see the skill level the kids are getting. Once the kids have practiced that fluency, typically any kid if he practices flow over and over again, he'll have a sense of what's going on in the passage, but not a deep comprehension. They then go on to the second activity, which involves them, first of all, taking a multiple choice quiz, where they often get at least two wrong because the questions all have distractors that are plausible, meaning they have evidence in the text. So they might get something wrong. The app then assigns them to a student that answered that question differently, and they work with that partner on coming to an agreement. They submit a new answer and an explanation for why they came to an agreement, and then the teacher sees this and can lead a discussion about why would a smart reader have put that answer and cold call on students. This is one of the things that's really the goal in a classroom: if you can cold call on a student that's not an incredibly strong reader, but they have in front of them the discussion that they just had with their classmate, they can say, “No, we chose A because in this part of the text, it was saying this,” and then really explain it to the class.
Then eventually the teacher sees where there are still misconceptions, can clear it up, and now the kids have a sense of that whole section—what happened—and they can go on to two other activities that ultimately culminate in them writing. So these four activities take about two days, and they've done it through the app, and they end with writing 150 words in 15 minutes in which they talk about a specific moment in the text and they explain why something occurred. And by the end of this two-day period, they really understand that section well and they can move on to the next section. They get through the whole book, developing a deep understanding that way. So we've scaffolded that complex text that way. Does that make sense?
Matt Pasternack: That makes complete sense. And so each kid, they have their own device sitting in the classroom?
Arthur Unobskey: Yes, they have their own Chromebook.
Matt Pasternack: And during that matching part, they physically stand up and go sit down next to another student? They move around the classroom for the conversation?
Arthur Unobskey: Yes, with their laptop. Kids don't have to have to bring these home, but they need to have their own at school.
Matt Pasternack: That makes an enormous amount of sense. How much teacher training is required to do it?
Arthur Unobskey: We have a four-hour introductory professional development in which we really focus on the introductory unit, which is a lot about how do you set up a classroom where students are going to get excited about not just the text, but about their own ideas about the text. So we actually start with personal narrative and we set up how to structure the activities, how to give feedback to students on their writing, how to lead discussions. That four-hour session is followed by four 45-minute sessions that are probably the first 12 to 14 weeks of the program, where they're learning how to introduce these different activities and use the software, both the data and how to assign things to kids, and why you assign certain things and how to lead the discussions that come out of that. And then we offer coaching for teachers. It varies how much they want or need that. And we're happy to. We check in with our teachers regularly, but a lot of teachers are able to sort of take off fairly quickly.
Matt Pasternack: What would be the total PD time for a typical teacher?
Arthur Unobskey: So a typical teacher, four hours and those four 45-minute sessions.
Matt Pasternack: It feels like you've been a little bit under the radar because I'm sure you're perfecting the product and the program. That being said, I would be curious distribution-wise, how many districts or schools are you in?
Arthur Unobskey: So, we're in four schools right now. We are in a school in Milwaukee, a school in New York, a school in Boston, a school in Maine. We expect to be in 12 for this fall. What we have wanted to do is to really make sure that we look at the data and work with the teachers really carefully in the development of this program. We want to grow at a deliberate pace. One thing that we are really excited about is the College Board is subsidizing the cost of this for districts, so it's low cost, particularly the first year, because they believe that for inner city or rural communities, it is uniquely effective because it gets kids ready for 11th grade work by 11th grade, even if they start at fourth grade level. The College Board has worked really hard to make more APs accessible for schools in under-resourced communities, but the passing rate has remained sort of stubbornly low, and so they see that kids are able to do that kind of work with our program. So we feel like we don't want to expand too quickly, but we do want to grow. One of the exciting things is we plan to work with schools in California, with charter schools there because they have been really responsive to the data, as well as some public schools, but we're excited about that prospect and we look forward to sort of partnering with a variety of different kinds of schools.
Matt Pasternack: You mentioned the College Board is subsidizing the costs substantially, but you're only going to 12 schools next year, so districts that are interested in participating, do they just go kind of on a waitlist or would you be open to increasing the number of schools?
Arthur Unobskey: We would be.
Matt Pasternack: One thing just looking at the description of the product online and then as you've described it now, it seems in some ways so obvious. I'm curious, why is no one else doing this? Or is there a parallel? I mean, many times in education it's like you can point to something—oh yeah, 20 years ago there was a program that did this and then it faded, no one knows why—so I’m kind of curious if you have any insights into that.
Arthur Unobskey: Right, so currently the conventional approach is to create a ladder of texts. To start kids who are at fifth grade level and you want them to get to ninth grade level text, you create a unit in which, embedded in that unit, are some fifth grade level texts, some seventh grade level texts, and some ninth grade level texts, with the idea that they'll get some background knowledge from the fifth grade level text, apply that to the seventh grade level text and do better, because background knowledge can help you with your reading, and then gradually build up that ladder to the point where they can read complex text.
There are two problems with that. One is if they don't learn how to read fluently those complex sentences with direct instruction, which just isn't being done. There’s really no place where there's direct instruction. There are fluency programs out there for adolescents, but what they do is they take elementary models—so they have kids read as quickly as they can and measure how many mistakes they make. Adolescents, don't put up with that. They have to want to do it, and they can't have a teacher telling them they're wrong all the time or they just will not do it. So you need to make it engaging. So that's one thing where they get stuck. They don't have that baseline level of skill development.
But the other piece is that we don't have time. These kids are in ninth grade. They have to get, by 11th grade, to be able to read grade level text in order to be able to enjoy, like, advanced biology. You can't read a high school level biology text and enjoy it unless you can read complex level text. So, we don't have time. And the only way to get better at reading complex text, because it's unique and different, is by reading complex text. So we spend all our time on it. What the other programs do typically is they get stuck at fifth grade level or sixth grade level or seventh grade level because the kids never learn how to read those complex sentences. And so what's happening is there's huge production of curriculum that has happened over the last five years and what they have done is created these units, and kids are not getting to the complex level text in those units; teachers are getting stuck. So, we’re different in that way.
Matt Pasternack: Definitely. It sounds very exciting. Is the plan from a sales distribution perspective to, once you get past the 12-school stage and once you grow more broadly, to just kind of compete for core curriculum RFPs in districts, focus on the specific subject?
Arthur Unobskey: Yes.
Matt Pasternack: One advantage that sometimes the incumbent publishers have is they're selling everything, so the economics for them are a little bit more favorable. It's harder to go single subject, although I think Zearn and some others have been quite successful in doing that. Do you have any thoughts on that or is it sort of wait and see?
Arthur Unobskey: Absolutely. Our approach is to build partnerships, like the College Board. We're trying to develop partnerships, particularly with assessment companies that also provide…there are a lot of different organizations that provide guidance to districts that are in trouble, essentially, or are starting new, or are…there's all sorts of groups that help, for example, charter schools, or schools in turnaround situations, and there's an element about “you need an approach for literacy” and that we really can provide that approach and really partner with those organizations. What we want people to do is come to this ready to do something different, not looking for the same conventional approach that's being used elsewhere. So, that's what we want, and typically when they get the news that they've got to accelerate learning very quickly, we want them to say, okay, I'm ready to try this different approach. I think through partnerships that's the way it's going to work. We don't anticipate, although we have some sales staff, we really anticipate doing this largely through people, through recommendations, and through working hand in hand with partners that are dealing with, particularly, results that require high accountability.
Matt Pasternack: Got it. I've got two more questions if you've got time for two more questions.
Arthur Unobskey: I certainly do. That's fine.
Matt Pasternack: Okay. Great. The first is, I think a lot of education entrepreneurs talk about wanting to continuously improve and have classroom results that are driving the product and there's this tight iterative loop, like you might see the big tech company that's looking at millions of data points or even many more, and the reality is a lot of times that tech tools just struggle there, right? I mean, they don't actually do the continuous improvement and that's what frustrates school districts so much because you just feel like, well, it's not actually moving the needle. My sense is that you have a different approach, that it's probably somewhat of your secret sauce, but hopefully isn't so secret that you can't enlighten others. If I'm right that you're really doing things differently on this, could you explain what you're doing that's different?
Arthur Unobskey: Right, so we talked earlier about what we had done with an organization that we created called the Writers Express. What we learned from that experience is that the best way of collecting data—and I think most people agree that formative data is the most instructive about skill development for students rather than one-time, high-stakes exams—is that if you create formative data that teachers can easily discuss and are actually just activities that kids do, you will be able to continuously improve. So, what we do is every activity that kids do—the fluency activities, the reading activities, the paraphrase activities, the analysis activities, the written activities that they do—all have an outcome that is typically binary in part, in which they either got it or they didn't. And that shows up, whether it's scoring or the analysis of the teacher on the writing pieces—they either got it or they didn't. The skill that they're working on is very clear. And then there's a qualitative piece that allows feedback to the student in which they get a sense of the impact they're having, whether it's the recording that they made, whether it's the activity with a partner, or whether it's the writing piece they had. So what enables us to sort of very quickly spot and improve our delivery is that we are able to see very quickly when student performance dips.
So, let me give you an example. In December of this year, December 2022, in our rereading app, what our students are doing is they're meeting with a partner who answered a question differently, and so if I put A and you put B, I have to explain to you and write into the app on my screen—and we’re sitting next to each other—why I wrote the answer that I did, and then you have to explain to me and I write what you say about why you chose your answer. And then we have to come to an agreement, and we have to write out where in the text finally we were convinced that one of us was right. So we've had an argument; we've chosen the correct answer.
What we did was, in December, we decided to lengthen those instructions because we thought that, for some of our students, it wasn't clear exactly how that conversation should play out. What we saw in three weeks or so is that the number of responses that we got from kids declined significantly, and we could see that on the app very quickly, and then we were able to cut out the wording that we used in the protocol and revise it so it was much shorter, and then the responses improved. So, it's very easy for us to see, it's easy for the teacher to see, the teachers were able to report to us that during rereading kids are not being as responsive, and we were able to look and see why. So I think that it's embedding assessment into what kids do, and what they do is actually a clear sign that they're able to do the components of skills that they need in order to read these texts and understand them. We're looking to add an artificial intelligence feature to the fluency, for example, so kids can get immediate feedback rather than the 24 hours that we currently have. There are other things that we're going to do with professional development, streamline it, but making the formative assessments not separate from what kids are doing is just incredibly important.
Matt Pasternack: Wonderful. Last question and then I'll let you go. I saw you put out a fairly controversial blog post, about high-dosage tutoring not being as effective with this population. You know, most studies will show how high-dosage tutoring is the most effective intervention, maybe one of the only effective interventions, and I actually fact checked it with some researchers and was just curious. I wanted to learn more about this, and one response was there just hasn't been as much high-dosage tutoring in some of these older student populations; there's not as much data on it. But I would be curious both from the research as well as from your own experience—you've lived this for a very long time—what has pushed you towards this current model and kind of away from high-dosage tutoring? Is high-dosage tutoring something that you could imagine incorporating into this at some point? My sense from the blog was not necessarily and I’d just love to hear a little more about that.
Arthur Unobskey: I appreciate that. So certain types of high-dosage tutoring has resulted in really good results in every place except for adolescent literacy. So middle and high school kids’ scores, if you look at the meta-analyses that have been done, tutoring across several years, there's clear data that it hasn't had much of an impact for adolescent kids in literacy. Now, they are trying, you know, the federal government and foundations and districts are putting enormous amounts of money trying to use it as an accelerant for kids because of the pandemic and, you know, they'll continue to research this. But I think that data will be consistent with past data. And the reason is because reading complex text requires an emotional commitment from adolescents. When we read sophisticated text, what we do is we go back and reread. And we kind of admit to ourselves that we don't understand what we're reading and we fix it. We go back and, what's called, repair it. And the research says that that attitude—it's a self-esteem issue, as well as an emotional strength, as well as a skill—that going back and repairing your understanding when you read complex text as an adolescent is the greatest marker for whether a student is going to grow in their understanding of reading. And if you do that, if you want to get adolescents to do that, an adult one-on-one with them is not nearly as effective as if they have their peers to support them in this challenge. And that's why they need a group of classmates that will both push them, pull them, sort of niggle at them, and do the kinds of things that only peers can provide. In that article you're referring to, I wrote about watching a child sort of say to himself, “I'm really bad at rereading.” And then basically, the only way that he turned around was not his teacher saying to him, “No, no, you're good,” which adults do, but his classmates saying, “Yeah and I figured it out—and here's the answer—by my rereading skills,” and the student realizing, “Well, wait a second. I can be just as good as that student,” and really improving as a result. Adolescents are complex. They have complex emotional things going on, and to go back and reread takes some strength. That's why I feel strongly that it has to happen in a classroom and you have to create that classroom community where those students are able to do it.
Matt Pasternack: When the pandemic hit—I mean, so much of this approach is grounded in being right there in the classroom and moving your seat to sit next to another student—did you just suspend operations when the pandemic hit early? Did you modify this for Zoom? Did you happen to find schools that stayed in session? How did you survive?
Arthur Unobskey: No, no. We definitely had schools that were out and, in fact, one of our schools was out more than the average because they wanted their—it was a K through 12—and they wanted their elementary in every day, so they kept the high school kids at home. Well, what we were able to construct, we had to construct specific ways for them to do these activities. But with breakout rooms you could do it. And it's not as satisfying. I think we've known about that. We've really learned that this time around that with all of our education, particularly for adolescents, they need the sort of in-the-air contact with their classmates. But it did work. You know, the gains weren't as good as they are in person, but the gains were much better than the average. So, you know, it is doable. Again, we hope not to go back there, but it's possible that we will and you need to make the adjustments.
Matt Pasternack: Got it. Thank you so much for your time.
Hear more from Arthur on his blog.
