What we’d like to build, and how: findings from our FOECast ideation week

From February  26th through March 2nd, a bunch of us conducted a week of brainstorming, discussion, and prototyping for FOECast.  The idea was to explore what a research project on the future of education and technology could look like.

Today I’d like to try summarizing our collective findings.

(For more information about the week and FOECast, check these posts and the FOECast website.)

(If you participated in the ideation week in any way, including just reading a few thinbgs, please complete our evaluation!)

tl;dr – people want a range of stuff.

 

There’s a lot to sum up, so I’ve come up with headers and themes.   To recap, we asked people to respond to five prompts:

1. What needs did the Horizon report meet?
2. What forecasting methods should we consider?
3. What shape should a new effort take?
4. What scope should this cover?
5. Make a prototype and/or proposal!

Responses addressed these, but also cut across the prompts.  There seemed to be a lot of agreement or consensus, yet also some… divides, or different positions along continua.  I’ll try to represent all of this.  First, I’ll identify framing concepts that would shape any such work.  Second, a list of specific projects and services people derived.  Third, a sketch of organizational thinking.  Last, a meta-note about a possible redirection.

Please, please correct or amend if I missed or mischaracterized key elements.

1. Frames

Community There was a general sense that any future project should involve and connect with a broad population.

Diverse and complex Several argued for a project to recognize that education and technology are very diverse.  Technology appears unevenly, as per William Gibson.  As Tom Abeles recommended, “the idea of a vision for the future of HEI’s will be heterodox”.  Audrey Watters called for attention to “race and place and status, for example.”  Steven Crawford reminded us that economics can strongly shape what technologies get used.

Many were interested in including a range of educational domains, such as K-12 as well as post-secondary education (kudos to Elena O’Malley).  People also called for covering non-academic learning, such as business training and informal learning.  Mark Ulett: “it is incredibly important to also take into consideration learning, training, and development.”  Sonja Strahl asked and observed:

if we look toward a body of “experts” to do the forecasting, do we look only toward higher ed experts? I think that the lines between higher ed and, if you will, the real world, are blurring a bit. I think there’s value in the “the real world’s” views being part of this project.

Description and prescription and politics Some ideation week participants (Taylor Kendal and Maha Bali, for example) (distinguished between describing possible or likely futures versus calling for preferred futures.    Others (Phil Katz, Audrey Watters) asked us to engage technology criticism, with a critique informed by political analysis.

Audrey also asked FOECast to be more self-aware of its own politics and power relations:

This sort of project is always shot through with questions of power and authority. There isn’t really any escaping that, I suppose. But there can be a better recognition of what that means and how it shapes the process and the outcomes – the decisions that are made based upon it.

Single or multiple? On the organizational level, some people argued in favor of a single, unitary project, along the lines of a given Horizon Report.  Others explicitly or implicitly called for a mega-project with multiple functions and projects.

On the content level, Audrey Watters asks us to consider if we want our project to produce a single, perhaps consensus forecast, or instead to share multiple futures.

Macro or micro? Some liked a top-level view, while others preferred concrete data about individual cases.  A similar opposition played abstract and conceptual information off of concrete data. Veronica Armour offered a fine synthesis:

It would be good to see an overall snapshot – think picture of the Earth from space that can people can then filter or drill down into based on interest or section of the industry (K12, HE, Library/Museum, training)

Education or learning? On the one hand, people were very keen on exploring institutional issues (change management, implementation).  We can call this education, as it focused on educational structures. On the other, there was interest in learning, the act of learning, which opens up the picture to informal learning, as well as on independent learners and other actors (scholars, writers).  Ted Newcomb: “I would like to see something that focuses on those of us life long extra-institutional learners”.

Continuous or punctuated production?  Some argued for single publications, while others argued for rolling or continuous work.  For the latter, we heard about frequent polling and wikis.  Anthony Helm brainstormed about having: “[l]ess frequency of the report, maybe every 3 years, but supplemented by deeper dives into specific parts of the report in the interim.”

Methods There was some discussion of methods, but not much consensus.  Indeed, Pat Tully asked us to think of first principles first: “What are those values and principles that are our lodestar, that guide us in the tools we use and how we use them?”

Not only technology There seems to be consensus that whatever project plan FOECast produces, it should include technology, but also address other domains.

Why is there a focus on technologies and tools, rather than solving problems? How can future trend reporting truly reach and cover a broad spectrum of how learning and development is evolving with “innovative” or forward thinking pedagogical practice?

Openness and transparency  There were many calls for these, and no opposition.

2. Desiderata

Edge.com-style questions and answers The Edge site regularly poses deep questions for a preselected group of relevant experts, then hosts their responses (for example).  We could do something similar – i.e., ask “How will mixed reality change education?”, then post replies from twenty VR/AR/MR gurus.

A survey of the community This goes way beyond the Delphi method’s focus group, and turns to the broader education and technology world.

The reverse funnel, also the concentric project This would be a group futuring effort in stages by scale:

…‘recognized experts’ using criteria that are published and refer to a mix of traditional indices (publication metrics, alt-metrics impact measures, standing in professional associations or whatever) combined with some more community-based consensus valuations of expertise. Who’s invited to speak at meetings we think are significant to this field or topic, for example. Then a next layer ring of expertise that might be derived from community ranking of nominated people.

A data trove Many asked for access to data about education and technology.  Some saw this in terms of individual technology implementations and experiments.  Others wanted a better sense of deployment (how widespread is the flipped classroom approach?).  One use case for this was institutional benchmarking (Kelly Walsh).

Data could be externally located.  Stephen Downes floated one model:

Analyze what bloggers and pundits (not Twitter or Facebook) are writing about, and identify key categories. Find examples of cases of instances of those key categories. Provide an index of what major industry consortia and standards organizations are focusing on. Filter patent searches for keywords or trends.

Shaping Tomorrow automates this kind of work.

Another way to organize data could be a web-based database, a la Drupal, upon which other things can be build (see below).

A showcase Laura Pasquini envisions evidence displayed in a certain way:

A digital showcase of applications beyond a webinar or webcast could include bit-sized examples of testing and experimenting with learning design, a technology in application for learning, or other via a podcast+show notes, video demonstration, testing exemplar of a concept, team blog of experimentation in progress, or a “behind the curtains” look for how to apply pedagogical practices.

Analytics Many wanted the ability to analyze data in various forms, from text responses to evidence of technology use.  At least one person recommended using natural language processing (one example).

Narratives Some participants expressed interest in narratives or stories about the future of education and technology.  At times this was from a consumer side (I’d like to read/listen to/watch/experience a narrative), while at others it was from a production side (I’d like to build a narrative).

Audrey Watters linked storytelling to multiple perspectives:

Can there be a completely different sort of storytelling in its place? One that isn’t a report? One that’s about sharing nascent counter-narratives and semi- speculative fictions, perhaps, rather than the shopping list-like surety of an industry-oriented forecasting document?

Leslie Madsen-Brooks was one of several voices describing multiple audiences for the Horizon Report.  Perhaps our new project would either be consumable by different audiences, or would include multiple stories suited for various populations (technologists, administrators, funders, etc.).  Peter Shea thought we “should acknowledge the complexity of prediction by creating several models of what might happen rather than confidently present one singular vision.”

Tom Haymes offered an example of this, by “divid[ing] the FOEcast into a Blue Sky Section and Mature Section.”

I think it’s important to stir the imagination with possibilities (even if they don’t pan out as foreseen) but also have a “mature” area that shows off technologies that are suddenly achieving widespread adoption. The real challenge for the practitioner is to connect the aspiration with the practical. This might be a key feature of the report(s).

Further, “Connecting Blue Sky with Mature is also a form of futuring as we can track the trajectory of technologies moving from one pole to the other.”

A workshop kit A document or product alongside the futures research, aimed at helping organizations translate it into practice.   Phil Long described a version of this he wanted to see for the Horizon Report:

 to leverage the HR in a focused translational way to the issues/needs of a given institution, or category of institution (R1 university, versus comprehensive…. whatever a useful breakdown of categories might be).

Visualizations One video discussion explored a variety of visualizations we’d like to see for this research, including heat maps, scatter plots, 4d imaging.  Daniel F. Bassill strongly recommend we publish maps, both geographic and conceptual.  Roxann Riskin offered another model: “I imagined your snapshot looking like an inverted ice-cream cone where participants can click into areas within the cone- zone of their environment?”

Environmental scan There was some interest in doing environmental scans, even micro ones.

Evaluation We need to build in a way of checking and improving futures work.  Several reminded us of the importance of checking one’s forecasts, a la Phil Tetlock.  Veronica Armour wants us to consider that “there needs to be a method for evaluating how we are doing. How will we know if we are identifying the “right” trends?”

Phil Long envisioned a more ambitious form of evaluation, using the scientific method:

we likely need to run some experiments to see how different methods turn out, their characteristics, where they do well and where the fail… [T]he only way to have any sense of this is mount some degree of parallel efforts to collect the data and see what outcomes emerge.

Scenarios There was some controversy here.  In favor, Phil Katz: “I like scenarios as heuristic devices, as long as people do not mistake them for predictions”.  Opposed, Stephen Downes: “I am also not a fan of scenario-based mechanisms because the core predictive work is conducted a priori in the definition of the factors defining the various scenarios. As well, it’s a bit like the meteorologist telling you that it might be hot tomorrow, or it might be cold, and maybe there will be rain, or maybe not.”

A version of scenarios comes from Cynthia Calongne/Lyr Lobo:

why not brainstorm “the most outrageous futures possible if technological capability was not an issue,” and then use the other strategies to envision what it would take to realize them? Rather than hire from outside, I’d crowd source it as a Great Thinkers” futuring exercise (free).

Ethnographic analysis Phil Long also suggested conducting ethnography about emerging technologies and their uses:

a metric looking at their adoption ‘patterns’ might be a valuable lens by which the judgment of ‘will this tech innovation work here?’ can be better made. I’m particularly thinking of the recent work at CMU by Lauren Herckis and the crying need for an effort to advance ‘implementation science’ (borrowing from Joel Smith’s term for this, with appreciation).

3. Organization

Some would like to see an organization arise that handles different functions.  Tom Haymes called for “a dynamic, thoroughly modern, and ultimately resilient network that can grow organically and provide immense benefit to a worldwide network of communities”.  Paul Signorelli developed a similar vision:

I would suggest interrelated “shapes” rather than a single “shape” (e.g., the printed/online Horizon publications). If we’re able to settle on a foundational central meeting place (e.g., a wiki, the Slack community, or something else that offers an easy-to-navigate structure), then our efforts could have rhizomatically expanding offshoots (interconnected blog posts; tweet chats that are captured and archived by whatever replaces Storify, reports/white papers that could be updated as needed; Shindig sessions along the lines of what Bryan so effectively does; an updated version of the reports so many of us have adored; or…or…or…)

This reminds me of Roxann Riskin’s call for “[a] next gen structure – a participatory structure”.

Stephen Downes offered a related but different view:

Organizationally it probably looks pretty typical, with an executive and a membership (but maybe with greater transparency and reporting requirements). I’d assign an editor to each ‘iteration’ (or each question, or whatever we would call it) who would curate the discussion. Analysis of the contributing discussions (via NLP or whatever) would be an ongoing responsibility (though various processes could be used). I think that the key here is this: for each thing that is done (an organization created, a topic covered, an analytical process undertaken) there is a ‘champion’ who will push that thing forward, in cooperation and with the support of the whole organization.

4. One strategic meta-note

Overall my sense is that participants really valued conversation about the future of education and technology.  They showed this by their various acts of participation, as well as many requests for community interaction.  Nobody cited a particular platform or venue for supporting this kind of discussion.  Some say the late Horizon Report as that kind of site.

In fact, Kay Oddone saw FOECast as a learning community, and that “Project FOECast is not only focused on the production of information for others – it in itself is a learning community”.  Maybe I’m rushing things too quickly by pushing for a project plan.  Instead, perhaps the most important thing to achieve now is a space or network for supporting these conversations.

What do you think?

And let me ask what you make of these findings.  Did I miss or mischaracterize any activity from the ideation week?  Which desiderata sound most appealing?  Do you like the frames?  What would you add?

(photo by duncan c)