The hope of the “Skills for the Future” collaboration is to make it easy for schools to treat so-called “durable” skills such as critical thinking, teamwork and perseverance the same as traditional subjects like math and English. That includes giving students new tests and a new report card that shows how well they have mastered those other skills as they apply to colleges or jobs.

The collaboration between the Educational Testing Service and Carnegie Foundation for the Advancement of Teaching started last year and added five states this spring — Indiana, Nevada, North Carolina, Rhode Island and Wisconsin. The Mastery Transcript Consortium which has already built a mastery-based report card became part of ETS, the company that runs the SAT and GRE college admissions tests, in May.

The partnership comes as some businesses edge toward skills-based hiring, rather than hiring for having a college degree. The partners also want to increase mastery or competency learning, where students progress at their own speed, rather than in lockstep with a class.

“This whole idea that education could be focused on durable and transferable skills is super exciting to me,” said Scott Looney, founder of the Mastery Transcript Consortium, which just added its innovative report card to the partnership this month. “It’ll make school more engaging, interesting for kids, but also make it more meaningful.”

“I think this is going to give us the ability to take this to millions of kids,” he said.

Carnegie President Tim Knowles said this new effort is a full reversal of what his foundation once promoted. In the early 1900s, the foundation popularized the credit hour or Carnegie Unit concept of measuring learning by hours spent on a subject. 

But Knowles said it is clear now that students learn at different speeds and the measure that share’s his foundation’s name doesn’t work anymore. ETS, Carnegie and others in the mastery movement want students to be rated on progress toward each skill, at whatever speed works for them, regardless of when a grading period ends.

Knowles said he hopes to replace the credit hours model “where bells ring between classes, where it’s time, not competency, that is the rule of the day.”

“Our aim is to build a new architecture that would actually enable competency-based learning to move from the edges of the profession where it’s lived for 100 years to the mainstream,” he said.

The end result, said Laura Slover, managing director of the effort, will hopefully be a way to show students’ character traits that many believe are just as important to success in school, jobs and life as academic knowledge.

“We are convinced that there’s a lack of social and economic mobility in the U.S., and that we’ve moved from a knowledge economy to a skills economy,” said Slover. “We want a portable transcripts or wallet, if you will, that shows where students are in their development, skills and abilities that they can use with employers, they can use to open the doors to college, and that are fair and reliable and meaningful for kids.” 

That shift, though, relies on schools to determine which skills to focus on and how to measure them. While there are many tests on subjects like English and math, there are no standard ways of measuring skills like communication, collaboration or digital literacy that carry across teachers, subjects, schools or states.

This spring, the five states that are dipping their toes into mastery joined Skills for The Future to help develop ways of measuring student progress on these durable skills. 

ETS said first steps included looking at the “Portrait of a Graduate” or “Portrait of a Learner” statements that — see Nevada’s or North Carolina’s   — that list the attributes and values they want students to have. The most common traits that will be the first  priority, said Knowles, are communication, collaboration, persistence, and digital literacy, with critical thinking and creative thinking close behind.

ETS, Carnegie and the states met in California in April to brainstorm approaches, with each state now meeting with teachers, students, colleges and businesses to develop ideas to pilot as early as January, 2025. A few early possibilities include interactive testing that adjusts questions – making them harder or easier or zeroing in on certain topics – based on student answers, as some online tests do now.

Some tests could be game-based, instead of just having students answer questions.

The partners are discussing how to use artificial intelligence and portfolios of student work such as papers, artwork, and projects created for multiple classes, as well as extracurricular or out of school activities to show character and interdisciplinary skills.

Portfolios can post a challenge for schools and for one of the long-range goals of the project – that skills can be reliably measured and believable to business or college admissions departments. Mastery schools use portfolios now, but those are dependent on subjective decisions by schools and teachers and are not verifiable to outsiders.

But Nevada state superintendent Jhone Ebert said Skills For The Future could develop guidelines that could be used in schools everywhere to create more consistency. 

Or some rating decisions may be left to teachers, but a third party may be able to offer a seal to be added to the new transcripts to offer some verification.

Ebert said she wanted Nevada schools, some of which have tested mastery concepts after a 2017 state law change, to have a role in creating new assessments.

“What has not gone well in the past is that someone just makes it a determination that this is the magic wand and how we’re going to measure everything,” she said. “Then it comes down and classroom teachers haven’t been involved, state leaders haven’t been involved. It is just this tool that has been made available and we are all going to adopt.”

“This process is much different,” she said. “We are all working together to co-design what it will look like and provide that feedback and input up front.”

The Mastery Transcript Consortium started in 2017 with private schools who wanted to show student progress from “developing” skills to “mastering” them. It built a transcript model that typically shows about 60 skills, as each school determines, instead of just the half dozen courses a student might take each semester. Some schools have used versions of it in college applications since 2019.

It now has 370 private and public schools or districts as members and says 500 colleges agree to accept students using the transcript.

It also has already built an intermediate report on student progress on durable skills that schools can use as a supplement to traditional academic report cards, if they’re not ready to make a full leap yet.

Though it keeps adding members, director Mike Flanagan said it is still a small group without the clout to take its work to a massive scale.

“For us to reach millions of students across the entire country on our own would have taken an infinite amount of time,” he said. “It’s virtually impossible. But ETS is one of the few organizations in the sector that has the scale and capacity to credibly help us reach millions of learners.”

Flanagan added that having ETS, Carnegie and five states joining the work  “lends enormous credibility to our effort.”

Whether this effort is enough to make mastery and measurement of soft skills take hold nationally remains  to be seen. Attempts to use mastery in states like Maine and New Hampshire never fully caught on and are often .

Among the pitfalls, he noted, are federal testing requirements under the Every Student Succeeds Act that don’t allow flexibility when students must learn some academic skills. And schools and parents could balk at a shift, he said, though Skills For the Future might succeed in doing enough advance work that it would be easier for schools and teachers to adopt.

“It’s a real challenge to do it well, but it’s good that somebody with the horsepower that ETS and Carnegie (have) are giving it a shot,” said Scott Marion, executive director of the nonprofit National Center for the Improvement of Educational Assessment. “Let’s not kid ourselves. If it wasn’t so hard, somebody would have done it.”

For any district, it’s difficult keeping students engaged all school year. However, when those students are spread out in 48 schools across an area the size of West Virginia, the challenge of ensuring continuity of learning feels even more overwhelming.

During the 2021-22 school year, half of the students in my district, Alaska’s Matanuska-Susitna Borough School District were chronically absent, requiring educators to jump into action so students could remain connected and on track to a successful future. By thinking creatively and merging policy changes with innovative ed tech solutions, educators can confidently help every student succeed, no matter where and how they learn.

First, we realized we had to stop the blame game and keep an open mind. The reasons behind attendance problems are as diverse as the students the district serves, and because of its location, Mat-Su has its own, unique issues. For instance, Alaska’s vast geography means that student athletes may be away at games three or four days out of the school week, and families often travel, hunt or camp for a week or two at a time. In addition, transportation constraints are an ongoing problem, as are inclement conditions ranging from snowstorms to earthquakes.

Until the pandemic, the district operated with the same attendance policy that had been on the books since the 1990s: Students with 10 unexcused absences a semester were immediately unenrolled. This essentially closed the doors on the young people who needed support the most — those with learning difficulties, instability at home and social, emotional and mental health concerns.

Today, online learning platforms offer the district a holistic view of each student’s grades, behavior and attendance, allowing school staff to easily identify red flags in attendance and use the data to prompt discussions with families and caregivers. Building these relationships has allowed the district to analyze the root causes of absenteeism and intervene with personalized solutions attuned to each student’s needs. 

Whether connecting families to wraparound resources that address mental health, transportation or trauma-related issues or working with staff and teachers to create a hybrid environment that allows students to learn both virtually and in person, the district strives to prevent a few absences from evolving into a chronic problem. We also no longer unenroll a student after 10 absences, but instead bring the family and stakeholders together to coordinate care in the event of a crisis.

Second, in its mission to ensure no student falls through the cracks, the district has taken a collaborative approach, developed by teachers, support staff and administrators, to clearly define learning standards and construct relationship-centered educational environments. This brings everyone together so we’re all on the same page.

Since the pandemic, the district has shifted away from the Carnegie Unit, which conflates time and learning, toward a more personalized, adaptive and mastery-based, standards-based system. While credits and grades are still identifiers of student progress, they’re no longer the primary criteria for evaluation. Educators have established standards in all subjects to ensure students’ mastery of knowledge and skills before they can progress to the next lesson. Through a combination of one-on-one instruction and virtual learning, teachers can help students revisit and hit learning targets they may have missed. As technology evolves, educators are working in sync to provide students with the academic support they need.

In addition, every high school student is enrolled in the district’s , which helps support their Credit, Career, College and Community Goals. Students are assigned an adult mentor who works with their teachers, counselors and families to help foster their success and growth. They also participate in weekly goal-setting and spend class each week working on reading, writing and mathematical skills. Because connectedness is a driver of regular school attendance, these relationships have been critical in engaging students in their schools.

Third, the district is integrating to align education with how and where a student learns best and to make sure that those learning from home because of transportation issues, illness or injury will receive the same enriching experiences as their peers in the classroom. 

All 19,000 students in pre-K through 12th grade receive Chromebooks loaded with educational tools that build on what they’re learning during the school day. In addition, the district has helped close learning gaps through guided online tutorials and the program, which assists with credit recovery for juniors and seniors. Taking full advantage of technology has been especially beneficial for rural schools that often have a smaller staff, fewer courses and higher absenteeism than their suburban and urban counterparts closer to the Anchorage metropolitan area.

Watching chronic absenteeism rates soar across the U.S. is incredibly disheartening for educators who want nothing more than to see students thrive. Until there’s a miracle solution that fixes every transportation issue, controls the weather and cures all illnesses, students will always be absent. It’s up to districts to move from penalizing students for missing school for reasons beyond their control to mitigating the impact of chronic absenteeism on their academic success. 

On this episode Timothy Knowles, President of the Carnegie Foundation for the Advancement of Teaching, joins Diane and Michael to discuss how this historic foundation looks to drive the future of American education. On K–12, they discuss why Carnegie has partnered with the Educational Testing Service and why they are seeking to assess a broader array of skills — not just focus on the standards that are already assessed. They also dive into Carnegie’s push to undo the Carnegie Unit and move toward a competency-based system. Knowles also shares details on the Foundation’s efforts to prioritize social and economic mobility in higher education by changing how they classify colleges and universities.  

Listen to the episode below. A full transcript follows.

Diane Tavenner: Hey Michael.

Michael Horn: Hey Diane.

Diane Tavenner: Well, we are fully in the holiday season at this point, and I’m super curious. A couple of clips away from the big part of COVID, are you noticing or experiencing anything different this year?

Michael Horn: Oh, yes, we are. We are hosting constantly, it seems. We have had one of my kids’ entire class and all their friends over. We’ve had parties galore, and it seems like it’s never going to stop. We’re going to do it apparently straight through New Year’s. So that feels like a big difference. As you know, we’ve been renovating our house. That’s basically done. COVID basically done. Knock on wood that there’s nothing else coming. And so there we are. And here we are in this, our fifth season, still working through some of the sticky issues in K-12  education, all the way into how it impacts higher education and lifelong learning, frankly, and trying to give people a different vantage point on how to think about these intractable — historically — issues. And I guess the last thing to say is, as listeners know, this year we’re doing a lot more guests, a little less of Diane, Michael, a little bit more of people out there doing some really interesting work. And today you have invited a guest, Diane, who is doing a lot of interesting work. 

Diane Tavenner: That could not be more true, Michael. It is my great pleasure to have invited Tim Knowles here today to be with us. He’s the president of the Carnegie Foundation for the Advancement of Teaching and Learning. And as you know, I am really privileged to sit on the board of that foundation. And so I have a really front row seat to the ambitious agenda that the foundation is undertaking. So much of what Tim and the team are seeking to tackle relates to the topics that you and I have been talking about on all of these seasons here, on Class Disrupted. And so I just thought it would be really fun to go back and dig into some of those, like, seat time, competency-based learning, assessment, accountability, but through the lens of a really historic foundation that has a really ambitious, modern agenda and has had really profound impacts on our schools that I don’t think most people realize or understand. And so I’m super excited for this conversation. Tim, welcome.

Timothy Knowles: Thank you. It’s a pleasure to be here.

Michael Horn: Yeah, well, we’re incredibly excited. I was really thrilled when Diane told me she was going to extend the invite. And before we dive into the work that you’re doing now that Diane just alluded to, I know that the Carnegie Foundation for the Advancement of Teaching and Learning has a long and pretty storied history. Can you tell us a little bit about the organization and why it has mattered to K-12 education in this country?

Timothy Knowles: Sure. So, Carnegie Foundation is 120 years old and it’s been instrumental to a wide range of educational things. The first thing it did, literally the first thing it did was create TIA, now TIA CREF, the largest retirement fund for teachers, professors, and people working across the social sector. It then created the pesky Carnegie Unit, or the Course Credit, the bedrock currency of our educational economy, which I expect we might get into a little bit further. And it’s done other important things through its history. It created Pell Grants, it created standards for engineering, law, medicine and schools of education. And more recently, it introduced improvement science, known colloquially as continuous improvement, to the education sector. But big picture, it’s an institution which has, or I like to think of it as an institution which has, looking around the corner in its DNA. It’s identifying levers to press, to improve both the quality of K-12 and the post-secondary sector, to incubate things, and to bring them to life at a scale that’s persuasive. And today our stake is firmly in the ground for first-generation underrepresented and low-income young people nationwide.

Diane Tavenner: Well, and that is one of the many reasons that I really appreciate being able to be on the board and be a small part of what Tim and the team are working on. The only thing that I would add is I was really surprised to learn when I joined the board that it’s the first nonprofit in America. It was enacted by Congress and became the first nonprofit in America. So, many of us who work in education, I think, take nonprofit entities and organizations for granted. And here’s the founding member of that team. So just a really fascinating, long, long history.

Timothy Knowles: I look really good for 120, don’t I?

Michael Horn: Better every day.

Diane Tavenner: Interestingly, for how old it is…Are you president number eleven?

Timothy Knowles: Ten. 

Diane Tavenner: I mean, not a lot of presidents.

Michael Horn: That’s impressive.

Diane Tavenner: Tim, you just alluded to it. For the last stretch of time under the previous president, because you’ve been here at the Foundation for a couple of years now, the Foundation was really focused on improvement science. And one of the interesting elements of this Foundation is that the current president really gets to define, has the full latitude to define the agenda. And so under Tony Bryk, that’s when I joined and when a whole vibrant improvement science community really formed. You’re continuing that. You believe deeply in improvement science and have a long history of it as a method for how we do our work, but then have layered this really ambitious agenda on top. I want to start with one of those meta outcomes. There’s a few of them that you’re driving to, and that is to accelerate social and economic mobility and achieve equity across the educational sector. And you just alluded to this. Earlier in this season, we had Todd Rose on the podcast and he shared a number of findings that suggest that a majority of Americans are really starting to question the ROI of four-year college and even our K-12 education system. And that they have this perception that education has become the end goal versus sort of a means to achieving a good life, economic security, freedom, however you want to say that. And this big outcome that you’re talking about seems to be in tune with the sentiments of the American public, if you will. So will you talk to us about why this big meta outcome is important to the foundation and honestly, what you think can be done about it?

Timothy Knowles: So I’m going to start with a sort of personal reflection about that. My first job as a teacher was teaching Southern African history in Botswana, and it was before apartheid fell. And so by day I taught a fundamentally emancipatory history curriculum, and by evening and by weekend, I was involved more directly in what was then known simply as the Struggle. I had the opportunity about 25 years later to visit South Africa, which I hadn’t traveled to, when it was free. And I met with artists and activists and clergy like Desmond Tutu involved on the ground in the Struggle. And to a person, literally to a person, they said it was teachers, students, and professors who broke the back of apartheid. From a personal perspective, if educators were responsible for that, our work here to accelerate economic and social mobility and achieve equity seems eminently doable. I guess I would also say personally that I want to live in a nation and I want young people to live in a nation. Whether you grew up on Navajo Nation or in rural Appalachia or in the South Side of Chicago, you have the opportunity, legitimate opportunity, to lead a healthy and dignified life. I’m much less interested in arguments about the particular kind of school you attend public, private, charter, home school, or the time it takes to finish high school or a postsecondary degree. I care much more about how to build systems that enable millions more young people to possess the knowledge and skills that they need to lead purposeful lives. I know for your listeners, there are some out there who are going to be persuaded more by data about why social and economic mobility matter. There was a study, just to cite one study, there was a study by the Federal Reserve in Boston and economists from Duke and the New School. It was called the color of money. And they looked at the net worth of families living across a range of American cities by race. And the average white family’s net worth was $247,000. The average Puerto Rican family’s net worth was $3,020. And the average non-immigrant black family’s net worth is $8. To be clear, I’m not suggesting education is not a powerful engine of economic mobility. We know it is. What I am suggesting, and where Carnegie is putting our stake, is that it could be a much, much more powerful one.

Michael Horn: Just, I mean your own personal story and how you come to this is inspiring. Tim, the few times we’ve gotten to connect at different conferences and so forth, hearing you speak about it always touches a chord, I think, for those listening. And obviously you just alluded to how you all now want to make sure that the system evolves and really creates a lot more opportunity for a lot of individuals. And I think that relates to a big partnership that has been in the news quite a bit lately, which is this partnership with ETS, the Educational Testing Service. Can you tell us about what you’re trying to do and why?

Timothy Knowles: First of all, I don’t think assessment is a singular answer to serving young people better. Young people need to love school. They need to be engaged. They need to feel challenged and pressed. They need to learn hard things and relevant things. They need to experience learning, not just enact learning. So I don’t think we’re going to assess our way to a better place. However, there are a set of skills that we know matter, that we know predict success in life, in the workplace and in the schoolhouse, and yet we haven’t paid them as much attention as we might. And their skills affective behavioral, cognitive skills like persistence, communication, critical thinking, creative thinking, collaboration. We think they deserve more attention, not at the expense of reading or algebra or history. Disciplinary knowledge really matters, and you can’t think critically without something to think about. But we think these skills in particular need to be elevated. We also know that these skills are developed in all kinds of contexts, both in the schoolhouse and outside, that many young people who demonstrate them, they’re too often invisible or illegible to postsecondary institutions, and to employers, and even to students and parents themselves. So just by way of an example of what I’m talking about, if I’m growing up in rural Indiana and I work for 2 hours every morning on my family farm, and then I get to high school at 7:30, every day on time. I have a 98% attendance rate. I do my homework on time, I get B’s or better, and then I have a job after school or on the weekends. Taken together, those skills, in my view, would represent persistence and they should be made visible to students themselves, certainly to educators and to postsecondary education institutions and employers. So if I was to state Michael, really simply, what we’re trying to do with ETS, we’re trying to build a set of tools that will provide insight into key predictive skills that the education sector has neglected. I don’t think teachers have neglected these skills, and I could say more about that. I think they know that these skills matter. But we want to build tools that will capture evidence of learning also wherever it takes place. And to make those insights visible and legible to students and parents, actionable for teachers, and useful for postsecondary institutions and employers. That’s at the heart of this.

Michael Horn: That’s super helpful. Diane may jump in as well because she’s been working in these domains for a long time. FoR1, I guess I’m curious when I hear you say that, from my perspective, critical thinking, creativity, things like that, there are a set of skills that can be applied in different domains, but being a good critical thinker is in a domain, right? It doesn’t necessarily cross unless you have domain knowledge. So I’m sort of curious how you square that circle with something like the example you used, perseverance, which I would put, in Diane’s language, the habits of success, different from skills, which might be a set of artifacts across lots of different domains to show those habits. And so I’m sort of curious, are you thinking of them all as the same set of assessments that will capture these? Or how do you distinguish some skills that sit within academic standards perhaps, or academic domains, let me say, versus those that maybe are a collective evidence across lots of bodies of work?

Timothy Knowles: That’s a great question. And frankly, is the work that we are doing right now is to figure this out in terms of which skills are we really going to draw on disciplinary knowledge? Which skills are we going to draw on extant data that may exist like the kid in Indiana I just described? And which skills actually do we need to build tools for from the ground up that we may not have a nuanced enough set of tools to measure, for example, collaboration or working with others? So do you need to build game-based or scenario-based tools that would help you, give you visibility in terms of how someone is developing on that arc? But it’s a very good question and clearly, whether it’s critical thinking or even persistence, you don’t want to divorce that from content and from subject matter. You learn a great deal about young people in terms of their persistence based on their approach to complicated problems and hard problems and how they go about solving them. So this isn’t divorced from disciplinary knowledge in that sense by any means. I think in terms of assessments, first of all, I should say the aim was not to take on the American assessment industry and all the politics that go with it and try to introduce an incrementally better set of disciplinary assessments that feels like that would be sort of a Common Core redux. And I think we saw that play out pretty clearly and we saw where dividends were paid and where they weren’t. So, I think really the intention here is to identify competencies that we know matter that predict success that are developed in all kinds of contexts and create a set of tools that won’t look or feel like traditional assessments and push the educational sector to attend to a richer array of outcomes. Another important thing that I think is worth pointing out, which actually makes me optimistic about this, perhaps more optimistic than I should be. There’s something, as you both know, but maybe not all your listeners know, that is sweeping the nation in the form of these things called portraits of a graduate, or portraits of learner. States and school systems and schools have been developing them, engaging lots of stakeholders, basically asking, who do we want our young people to be? What do we want them to be able to do? So colleagues from ETS analyzed as many as they could find. This is one of the wonderful things about being partnered with ETS. I feel like I have 3000 new employees I can ask to do things. But they analyzed all of these portraits, and there were about eight to ten core skills that Americans say they want young people to possess upon completion of K-12. It’s almost as though – and this resonates, Diane, with some of your work – but it’s almost as though there’s an invisible consensus about the core purpose of schooling. Kind of a river running through our nation, whether in red places or blue places, in cities, in rural areas, about what we want our young people, who we want our young people to be. That’s hopeful to me. So if we can help the other thing that people say about the portraits, if you speak to them candidly, is A) They haven’t changed anything, like we haven’t actually changed what’s going on on the ground, even though we put a lot of energy into it, and B) We have no way of measuring these things. That, to me, represents an opportunity in the US, right now, that I think is worth plumbing.

Michael Horn: I’ve just learned a tremendous amount from you, and I had a takeaway that I think I haven’t had from the press stories on this, which is, in essence, you’re not trying to do what we recommend you never do in disruptive innovation, which is to try to leapfrog the incumbents with a better assessment or a better this widget whatever, but instead go to the areas of non-consumption where the alternative is nothing. And you’re right. I see the same thing in the portraits of graduate, which is there’s no teeth. There’s no way to measure or represent or have an asset based framing around these things because there’s nothing to measure them. So you’re going there. I think maybe the second question is less mine and more what I think a lot of people are wondering, which is why partner with ETS on this? Because they have a reputation in different quarters and different ways, as you know. 

Timothy Knowles: That is a completely fair question, Michael. And I know you both know as well as I do that most assessment companies across the world are grappling with what their future will look like and are seeing, quote, market share evaporate really quickly. Standalone assessments that bring schools to a screeching halt for two weeks in May and are not predictive of very much, I hope are not going to be part of the equation for the long term. And yet those very assessment companies, including ETS, have made an incredible business based on that design. ETS is clear-eyed about that, in my view. They hired a new CEO, Amit Sivak, who is exceptionally clear-eyed about. And one of the magnetic forces, from my perspective, was they have the capacity to build for scale. I don’t, Carnegie doesn’t. We’re a small organization. When I introduced to the board the idea of focusing on the future of learning, which is really the aim here, is to get at learning. One of our board members, who is a very well regarded scholar of assessment, said, well, what about the future of assessment? And at the time I thought, we really don’t have the capacity to build credible, reliable, valid tools to do some of this work. Then, Amit, who I’d known prior to ETS, joined ETS, and I thought there was an opportunity that led to a year’s worth of conversations about whether they are willing to really try to innovate and in essence create a separate entity within ETS, but with its own walls and autonomy to build a new set of tools that would attend to these skills, that would think about assessment in very different ways and that would be focused on the insights that were generated, not focused on the test as it were. So that’s why ETS. Now, to be fair, again, I think the test for us is can we build something different? Is it going to be useful to young people? Is it going to be useful to parents, to teachers? I think we can, but I know we won’t know unless we try. That sounds slightly glib, but I think it’s true. Like we have to take a shot at broadening the picture of what we say is important for young people. It bears probably saying that we met recently as part of this work with the 50 teachers of the year from across the country, from each state, and introduced the work to them. And literally there were some teachers in the room in tears and I was like, “Why?” But they were saying, bring it. This is the work we want to do. This is in essence the work that parents know we should do. And this is why we started to teach in the first place. That’s my short answer to “Why ETS?” We have enough elegant examples that live around the edges of our profession. Everybody in this sector can point to elegant examples of competency-based learning that haven’t scaled. So we need to think about – if we’re serious about tipping or using this tipping moment – we have to figure out how to enact at a broader scale than we have tried to historically. 

Diane Tavenner: I will just add here because I hear the critiques, just like you, and the questions. And I will just add from a personal experience, I think you might know this, Michael, and Tim, you certainly do, that several years ago, Summit actually partnered with a startup assessment company that was doing these exact types of assessments. So I know they’re possible, I know that they can be done. And then, of course, as a startup company, they got acquired and employers valued and wanted these types of assessments and they couldn’t stay in K-12 where the market was so competitive and unreliable, etc. And that was such a disappointment to me because I saw such the possibility of those types of assessments and how they could be used and that they really were possible. And so it feels like this is where the sort of solidness and the expansiveness of ETS, perhaps, enables us to move forward. And I would just add a fun fact, which is I don’t think relevant, but ETS is yet another entity that the Carnegie Foundation created and then spun out.

Timothy Knowles: We did — 75 years ago 

Diane Tavenner: Tim, you have started alluding to this already because these things are all connected and linked, but you said assessment is just a small part of it. And when you first started, it wasn’t even a thing that you were thinking that we needed to do, because what you’re really setting out to do is sort of build this architecture that produces what you call reliably engaging, equitable, experiential and effective learning experiences for all young people, every single one of them. And I think that those words, those concepts describe the type of learning that Michael and I are talking about all the time, that we are advocating for, that we believe in. So beyond assessment, what does that architecture look like? What else is happening to try to bring this to life?

Timothy Knowles: Here, we need to move away from models of schooling singularly dependent on the Carnegie Unit or the credit hour. It was established in 1906 to standardize an utterly unstandardized educational sector. So it was a great plan in 1906. But since 1906, we’ve learned a great deal from learning scientists and cognitive psychologists, neuroscientists about what knowledge is and how it’s acquired. So we need learning modalities that are truly competency or mastery based, whatever the language you want to use, that allow young people to solve real problems, that support experiential education, that enable them to work with mentors and experts and peers. The problem is not that we don’t know what this looks like. We do. Again, we can all point at examples of it. The problem is we haven’t figured out how to bring it to life at a scale that’s persuasive. Thing one for me is building, in essence, existence proofs and networks of existence proofs and amplifying and elevating them because this work is happening in ways that will generate momentum and attention. And I think we’re in an interesting moment where I’ve talked to 18 or 20 states in the last four months. State leaders, state chiefs, governors, they’re interested in how do we move to competency-based systems. There’s are opportunity windows open at the school system and state level, I think, post-pandemic that we have to leverage. And part of it is about cracking the Carnegie unit. Second thing I’d say is, and you may laugh me out of the podcast, which might be a first to be laughed out, but we need to think hard about learning experiences or curricula. And I know people feel like they’ve been down the curricular road before, but the tools and supports for teachers and students have to be taken into more careful consideration. The problem with the wave after wave of standards and accountability efforts over the last 40 years, and this is completely oversimplified, is that we thought if we cranked up the standards and tested for them on the back end, that somehow magically in the middle, the work that students and teachers would do every day would change. And I think the sort of governance reforms that led to charter schools were not that dissimilar. The theory being if we provided schools with flexibility and autonomy over hiring and money and use of time and governance, somehow the stuff that kids did every day would shift and we didn’t see that really occur. Part of the architecture demands building learning experiences for young people across disciplines, which are course-based, which are unit-based, which can come in different sizes to use that language, are much more engaging, much more experiential equitable and effective. So first thing is the Carnegie unit. Second thing is actually what gets taught. And the third thing is policy. The Carnegie unit has infiltrated much of our state-level policy, and I think we just assume that perhaps the states provide waivers so people can do what they want. Well they don’t. Seat time is the rule. That is the rule. Mastery or competency is not the rule. 990 hours of instructional time per annum, or some variation on 990 is the requirement for the vast majority of states. I’m a fan of guardrails, so I understand the argument that, “Well, you want to be careful about removing the guardrails.” But I’m not a fan of guardrails that don’t acknowledge what we’ve actually learned about learning over the last hundred years. And that’s the peril with this singular devotion to the conflation of time and learning. In my view, there’s a set of policy opportunities, if I was going to frame it in a more asset-based way, that I see. And there’s an appetite. And again, red states, blue states, both are interested. This is oversimplified, but I think the majority of the more conservative states that I talk to are interested in employment and access to jobs for young people who may otherwise leave their state. In the blue states, the interest is more about access and opportunity. But I think both are the same in this case, they’re fundamentally the same. Access and opportunity is really about employment, is really about social and economic mobility. I think there’s some more common ground, despite the kind of thrum of our national political discourse.

Michael Horn: I think you’re right. And I get super excited when you start talking about replacing this time-based unit – from the foundation that put it in place – with something much more meaningful and meaty. And it’s not surprising to me when I hear you – I want to use the word preaching – about this wisdom that you had to go and that you have. 

Timothy Knowles: Ouch

Michael Horn: Well, I want to yell “Preach!” But when I hear you say, “We ended up having to go to assessment,” that makes sense to me, because you have to replace the unit of time with something that is measuring progress in a different way. And so that makes sense. Now to switch gears completely, though, another part of the work — you’ve got your tentacles in a lot – another part of the work that you all do, and something that Diane and I have been talking a lot about on the show, is higher education, of course. And you all have a profound impact about how we think of the categorization of colleges and universities in this country. And you’ve made some big moves to change that. For our listeners that are less steeped in higher ed, can you tell us what the Carnegie classifications are in the first place, why they matter, why they have mattered, perhaps in the way that was not intended, and what you’re doing now with them to change those incentives?

Timothy Knowles: One of the things we do is we classify every postsecondary institution in the nation, almost all of them. There’s some that don’t submit data to the federal government, and so we don’t classify those, but something like 4500 institutions, we classify. Many of your listeners or some of your listeners may have heard of one of these classifications “research one” or “R1” classifications that comes from us. That spawned an arms race in terms of higher ed institutions aspiring to be R1 institutions and designated R1. Not just because of the One, but because the federal government follows it up with vast tranches of capital, of public capital. So there are real incentives to become an R1 that led to this arms race. So when I arrived at the foundation, the classifications had basically been spun off and had gone through very modest changes for 50 years. So since I got there, we’ve brought the classification…I’ve invented a new term, it’s called spinning on. We spun it back on and we brought them in house. Now with our partner, the American Council on Education, we’re trying to reimagine them from the ground up. So in 2025, all postsecondary institutions in the country will be classified in new ways. There’s lots of vectors of the work here, but one thing that I’m particularly excited about, and I hope will resonate with the kind of work we’re interested in on the K-12 side is developing a classification focused on the extent to which postsecondary institutions are engines of social and economic mobility. So every higher ed institution in the country will receive an economic mobility classification. So classification is distinct from a ranking. We’re not of the view that you can distinguish in credible ways between an institute number 599 and 600 on a list. Classifications are groups of institutions. So like institutions, in that sense, we’re less interested in naming names and creating another rank order. The primary aim here is to learn what institutions are doing to effectively accelerate social and economic mobility, to develop public policy that supports it. And just as R1s have been the recipients of large tranches of public capital, to drive public capital to those institutions that are accelerating economic mobility. So that’s that body of work. It’s fascinating because the big world doesn’t know much about it, but the higher ed world pays extraordinarily close attention to it. So two weeks ago I had a conference call with 1500 higher education leaders. That’s a third of them, or something close, which suggests how closely they’re paying attention. So we want to draw attention to one of the things that I think makes America and higher education great, which is the extent to which they’re actually making improvements in terms of young people from low-income backgrounds, first-generation young people, and underrepresented young people in particular.

Diane Tavenner: Yeah, it’s really fascinating. It’s so interesting that a tool like that is visible to everyone. I mean, so many of the national rankings are based in part, like, if you look at their formulas, the beginning of the formula is this classification. So we all see it, but we don’t understand where it comes from. Super hopeful about the potential impact there. Okay, I have to squeeze one more thing in here before. This is like the speed round. But when I was in grad school, I learned about the Committee of Ten and the profound impact that they had. I’ve talked on this show about this before — Michael and I have talked about this — about how they really defined what the order and sequence of high school curricula was and put the sciences in order, alphabetically biology. So we did it that way for a really long time. You have launched something called the Carnegie Postsecondary Commission. So people should not be surprised to know there was a relationship with the foundation and that old committee. So you’ve launched a new commission. Tell us about it quickly.

Timothy Knowles: So sure. The Committee of Ten was founded in 1892. It was chaired by a guy called Charles Elliott, who was the president of Harvard at the time. Interestingly, and I didn’t actually know this until recently, Charles Elliott was charged by Andrew Carnegie to establish the foundation that I’m responsible for. So the congressional order that says we better create a nonprofit for this thing, the first signature on that congressional order is Charles Elliot. So it’s a very tangled web that we live and weave. So the postsecondary commission is a group of not ten, but seventeen K-12 and postsecondary leaders. My hope is that they become the Committee of Ten for this century that will be thinking hard again about the question of mobility and how we create not just K-12 and post secondary systems, but systems that might even become much more blurred. So K-16, K-to-work systems that are going to not try to reach consensus as a group, and they all signed up with this agreement. The aim is not consensus. The aim is to develop action papers that will provoke both thinking and policy, certainly, but then to help shape the work of the foundation, particularly on the post secondary side for the next decade for what I hope is my tenure. It’s a commission with institutional engine underneath it. It’s an extraordinary group of people. I won’t name them, but I would urge anybody who’s interested to go and look at our website and meet them because they are almost, to a person, first generation leaders who are doing exceptional things ranging from running large public systems to small colleges to K-12 systems serving young people who depend on the quality of school the most. It’s an extraordinary group. We just convened earlier last month, and the world should get ready.

Michael Horn: Well, with that tease, why don’t we leave the conversation there from a work perspective, but before people tune out, Tim, you’re joining us. Diane and I have this end of show segment where we talk about things we’re reading or watching, and we try to make them not about our work. We don’t always succeed, but we try. So can we ask you what’s on your watching, reading, listening list?

Diane Tavenner: Sure.

Timothy Knowles: I have a weird tradition. I read poetry from December 1 to the New Year because it makes me think differently. So, I’m right now, who am I reading? Haki Maributi, South Side of Chicago poet. Gwendolyn Brooks and W.H. Auden, not a South Side poet, so a mixture. But I find it takes me out of my day job and makes me think about the world and people and what I’m here for in different ways.

Michael Horn: I love this because poetry is one of those things I always wish there was time for. I never know how to fit it in. You may have just given an idea for not just me. So, Diane, what’s on your list?

Diane Tavenner: I’m going to go a little bit different this week. Coming off a time period where we had lots of family and fun friends around, I did a jigsaw puzzle this past weekend. Some special guests dropped in and helped put a few pieces in. It was so much fun. Makes your brain think differently. Very social. So that’s my, whatever, enjoyment of choice this week. How about you, Michael?
Michael Horn: I love that. That feels very COVID, I will tell you that, but I love it. Mine, I will go, I just finished the first season of The Morning Show with Reese Witherspoon and Jennifer Aniston and have moved into season two and really enjoying it. It’s a complicated set of storylines that follow a little too closely, like real life in 2019–20 and so forth. And we’re getting into the COVID period right now, but it makes you think, it makes you laugh, it makes you cry, and it’s enjoyable. So that’s where I’ve been. And we’ll wrap it there. Tim, huge thank you for joining us, talking through all the initiatives that you all are doing at Carnegie. And for all of us, we will stay tuned. And for all of those listening, we’ll see you next time on Class Disrupted.

The new tests, to be created by the Educational Testing Service and the Carnegie Foundation for the Advancement of Teaching, are meant to usher in competency-based forms of schooling that would allow students to proceed through academic material at their own pace. Leaders of both organizations hope they will also capture a broader array of non-cognitive qualities, like teamwork and relatability, that are highly prized in the modern workforce but undetectable through conventional academic metrics like grade point average or school attendance. 

The adoption of more personalized instruction and assessment has faced a key obstacle in the form of , the namesake foundation’s strict definition of annual credit hours that students must accrue to demonstrate their grasp of material. (The calculation essentially breaks down to one hour of seat time per day, per subject, for 24 weeks.) Though largely unknown outside the education world, many high schools and universities have based their academic requirements on the Carnegie Unit . 

But Timothy Knowles, the foundation’s president, said that while the Carnegie Unit had served a useful purpose at one point, new discoveries in neuroscience and cognitive psychology have proven that pupils learn different subjects at highly variable rates. What’s more, he added, the capacity now exists to test for valuable qualities that were previously invisible to admissions officers and employers.  

“We’re in a position to do something that we hadn’t before,” Knowles said. “Unlike 20 years ago, we can actually reliably measure the skills that we know are predictive of success in postsecondary education and work.”

Competency-based learning and assessment has long been theorized as a preferable alternative to existing educational models, which critics describe as too standardized to deliver instruction to individual students with vastly divergent levels of academic preparation. Instead, they allege, the status quo came to reflect the production processes of 20th-century industry, with students replacing widgets as the product. In , Knowles himself telegraphed his desire to phase out the Carnegie Unit, calling time a “crude” metric to determine educational attainment.

With a range of philanthropic and education-focused advocates backing the movement, has promoted some version of competency-based policies. Those efforts hit in Maine, where high school graduation requirements were refigured over the last decade to emphasize proficiency on subject material. But disputes over the definition of proficiency and teachers’ differing grading standards led many to question the new approach, with legislators later backing away from the competency-based model.

Similarly rocky transitions were seen in and , which attempted similar shifts. The central puzzle facing critics of the current model (i.e., calendar-centered requirements and standardized assessment) is what will come to supplant it. 

Scott Marion, president of the , said that the challenge in executing the hoped-for switch to competency-based learning lay in designing realistic measures of achievement to replace existing tests. To deliver on advocates’ promises, he observed, such measures would need to be both tailored to individual students and academically credible.

“Competency-based assessment is not for the faint of heart,” Marion said. “It’s being done quite poorly in a lot of places. So if ETS and Carnegie can bring a little more rigor to it, it might be good.”

With interest in competency-based approaches growing, more players have leapt into the field, with the best-known among them that developed a “mastery transcript.” The project has gained adherence among high schools over the last year.  

The ETS-Carnegie proposal is also emerging at a time when traditional high school admissions exams, such as the SAT and ACT, have lost significant market share. Both the aftereffects of the pandemic and concerns about inequitable outcomes from standardized testing to go test-optional in the last few school years. With those leading indicators of secondary achievement potentially passing from the scene, demand is expected to rise for measures that could take their place.  

ETS, which administers the widely used GRE, PRAXIS, and TOEIC tests, during and after the pandemic.

Perhaps the biggest question hanging over the newly announced partnership is the proposed measurement of not just cognitive and behavioral skills — including everything from comprehension of math content to teamwork and leadership — but so-called “affective” skills as well. As described by ETS head Amit Sevak at the educational technology conference ASU-GSV, such skills could include something like emotional intelligence, or the ability to successfully convey sincerity and empathy to others. Just how those kinds of competencies can be conveyed to students, let alone measured by third parties, is debatable even to backers of competency-based instruction.

Michael Horn, a cofounder of Harvard’s Clayton Christensen Institute for Disruptive Education, said he would be watching the development of such measures carefully.

“This part, from my reading of the literature on assessment, is both unproven and underdeveloped. So the how is going to be very important,” Horn said. “I’m going to be very curious to see what the investments look like as they go forward, and I hope they don’t overpromise.”

While no concrete timeline has been released for the conception of the new suite of assessments, Carnegie and ETS to conduct a multi-state pilot that could begin as early as next year. In an interview, Sevak said he envisioned students being able to access a digital “transcript” detailing their ongoing growth in areas like collaboration and creativity. Real-time data could build their awareness of their comparative strengths and weaknesses, he added.

“That more holistic approach is in contrast to much of the assessments in K–12 and higher education, which are really cognitive-driven and tied to logic and reason,” Sevak said. We’re looking at a more holistic approach that is more tied to the future of work.”