one observation i’ve made while exploring participation in challenges is that there are many challenges that are interesting to read but i don’t see the point or value in completing the specific tasks as laid out. i end up wanting to pick and choose from the items– and i do. this suggests why i think many people never finish challenges. the pattern is, we see a challenge where the first few tasks appeal and are accessible to us, so we follow/adopt the challenge, and then the later ones don’t resonate as much, or aren’t as easy, so we don’t complete them. participation peters out.

we are often designing tasks within challenges to become progressively harder, but challenges aren’t meant to teach you stuff (that’s what groups, and the web, and life, are for), so even if in the best case scenario your challenge inspires people to go off and do or learn something new, you’re still basically losing your audience at that point.

both of these issues relate to a key assumption that i am beginning to feel is wrong: that we need to keep people’s attention. learning is an incredibly personal thing. the best way to offer flexible learning pathways is to make each discrete learning unit as small as possible, allowing the learning path to evolve and iterate. vanessa refers to resnick’s learning spiral, which iteratively revisits the steps imagine --> create --> play --> share --> reflect --> imagine .... one way to see challenges is as discrete opportunities to demonstrate learning at each iteration through the cycle.

one approach to this would be to have single-task challenges. these could be larger in scope, and correspondingly more open-ended. for example, focusing on the output but not the methodology, or the methodology but not the output. being more open-ended would encourage a broader diversity of approaches. it also scales better because it creates less work for the challenge mentors: with the Twitter API challenge, i spend a lot of time ensuring that everyone who posts to the first “introduce yourself” task feels welcome. but most of that work goes unrewarded, with most people never completing the subsequent task (it’s fairly clearly an exponential falloff rate).

with a single-task challenge, mentors get to spend their time interacting with peers about their ideas and content. and importantly, there is no expectation by either the mentors or participants about retaining participants’ attention. the main difference from our current approach is that the unit of accomplishment and recognition is tied to this single task.

a practical example of my own participation waning came up when i decided to adopt vanessa and audrey’s exciting writing for the web challenge. i like writing for the web, and the idea of having a community of peers around doing that. i think i could show that i have done a certain amount of writing for the web and have basic proficiency at it. but some of the specific tasks laid out in their challenge interest me, while others do not. or maybe it just happens, for example, that i don’t have a blog post i want to critique. can that tell us something about challenge design?

the problem, of course, isn’t that writing a critique is a bad exercise. it’s that it’s one of many possible exercises that would be equally appropriate. but because challenges are broken down into tasks, we tend to make them extremely prescriptive. and yet we know– in fact, arguably p2pu is founded on the idea– that there are arbitrarily many paths to a given level of proficiency. this suggests that we should be experimenting with less prescriptive challenges, challenges that are more open ended, and expecting more from our peers.

as kimo said in there’s no speed limit “the standard pace is for chumps - the system is designed so that anyone can keep up.” when i designed the twitter api challenge i definitely fell into this trap. i was actively trying to ensure that anyone could keep up. that’s not bad in and of itself; what’s bad is that our notion of “keeping up” is rooted in a system where inspiration is the exception, not the rule. the result is that learners are babied and the challenge is less fun. it turns the participants into consumers instead of creators, and we reinforce the very cycles of dispassionate, passive learning we are trying to move away from.

on the other hand, when learning new stuff on the web i definitely seek out detailed summaries and step-by-step introductions. it’s not that providing detailed breakdowns is a problem. it’s the assumption that those breakdowns should map in ANY way onto actions taken by the learner. that is, the problem is that we turn breakdowns into prescriptions. this limits our learning material and processes to introductory levels of learning, limits the sophistication of ideas we can achieve together, and correspondingly limits our community, too.

in summary, my takeaways after working with challenges for a few months are the following:

1. create single-task challenge content
2. separate action from consumption: absolutely do create rich and detailed learning materials, but offer it as guidance instead of prescriptions.
3. create a cultural emphasis on mentorship and recognition

i’m feeling pretty excited about these conclusions, and have gone ahead and tried a new challenge in the single-task format. the third item on the list above now becomes even more important, because it keeps the community element focused squarely in the group aspect of p2pu– which is exactly what it was designed for.

Microdata appears to be a middle ground between microformats and RDFa. It’s not totally clear if RDFa and microformats will ultimately be inter-compatible, and there still seems to be a lot of debate about which one to support, if both should be supported, both changed to support each other, etc.

In the meantime, they’re not actually that fundamentally different. I’d like to start working on the approximate markup for capturing basic science objects within a webpage. We should be able to represent the key objects in both formats.

General development goals:

• Agree on some core open science objects
• Put a spec together
• Develop creation tools (markdown syntax, TinyMCE, wordpress plugins, liquid tags, etc.)
• Provide a validator
• Develop consumption tools which can read the markup
• Develop discovery tools and a reference website which can injest and cross reference between these objects

This is a post with some thinking about how to get started, and some specific ideas. It’s all exploratory at the moment - a work in progress.

Features and Implications

All objects should have timestamps so they can be ordered temporally. They should also have license and, optionally, attribution fields. It might be interesting to experiment with self-certifying (ie, secure) research objects (cf. zooko’s triangle). You can’t ever prove that a similar idea has not been proposed before, since that is a function of the absence of information. But if you make the url for the research object into a hash that includes author and content information, then someone couldn’t for example copy the object and just change the author field. They could do this and change the url, but then they would break links to that object, which would show that it was a) new and b) not the same as the object others were linking to. It’s not perfect, but it’s something.

Using timestamps associated with discoverable hypotheses and questions, you could imagine similarity analysis identifying otherwise unknown, or early instances of an idea, tracking progeny, or even calling attention to plagiarism.

The Mozilla badges project uses the notion of issuers. i’m not sure that’s quite what we want in the science world, but imagine a service for registering certain ideas and pieces of work. Almost like a certificate authority-type role for science, where anyone could play that role, as long as the people hosting their data with that CA trusted them. Interesting, though not sure how I feel about this. Does it encourage openness and reuseability by giving people a means to register their work? Or does it raise the barrier to participation by adding unnecessary complication?

Example science markup object: Question

Following the current (draft) microdata spec, markup for a question object might look as follows:

1
2
3
4
5
6
7
8
9
10
11
12

<section itemscope itemtype="http://example.com/microdata/science-vocab/question"
  itemid="hash value" >
  <meta itemprop="research-topic" content="human computation" />
  <meta itemprop="author" content="Jessy Kate Schingler" />
  <meta itemprop="license" content="cc-by" />
  <time itemprop="created" datetime="2012-03-13"></time>
  <meta itemprop="tag" content="tag 1" />
  <meta itemprop="tag" content="tag 2" />
  ...
  <meta itemprop="tag" content="tag n" />
  <span itemprop="content">Is the sky is orange?</span>
</section>

This is basically a set of key-value (itemprop-content) pairs, with a reference url for the science objects vocabulary. the itemid hash value, which is optional, could be computed as a self-certifying hash, either by the local authoring tool/plugin, a javascript library, or returned via a callback. (I suppose we’d need to either specify the hash function in the spec or the object would need to indicate the hash function used).

Pretty much all the fields above could be optional. The minimum required set of fields might be much more minimal:

1
2
3

<section itemscope itemtype="http://example.com/microdata/science-vocab/question" >
  <span itemprop="content">Is the sky is orange?</span>
</section>

Is this too minimal? What are the minimal set of metadata fields needed to make something useful? At one extreme, the basic goal of discoverability is supported as long as something is marked up as being a question. Our primary goal is achieved. On the other hand, it has no other identifying information… My personal leaning is towards strongly encouraging basics like license and author, but that the basic marking up of something as a question is sufficient for it to be identified and composed by other tools, and is thus sufficient. What do you think?

Making things easier

WYSIWYG editors make authoring content easier and faster. For example, one could imagine a liquid tag for the question object:

1
2
3

{% question topic="human computation" tags="one two three" %}
Is the sky is orange?
{% endquestion %}

This could be expanded by a piece of javascript or a framework-specific plugin. The javascript option could pull in user-specified defaults for any of the fields, like license and author. There could also be services that host content-creation profiles, or people could host their own. (could these be added as openID profile attributes?)

Design Implications

Science markup should be easy to use by a lay user, and low-enough overhead to use informally for a wide variety of relevant content and publishing venues, from online journals to blog posts.

In terms of design, we should think about how we want to use marked-up science objects. It’s not about semantically encoding every last component of an idea. A little structure goes a long way. Building links between content helps with discovery and facilitates greater re-use. It supports needed re-examination and redefinition of credit, re-mixing, and access, and the ability to recognize, relate, fork, and credit scientific content on both a more formal, and a more fine-grained, level.

Other references:

• An Uber-comparison of RDFa, Microdata and Microformats
• RDFa Lite 1.1
• HTML Data Guide
• Microdata on Wikipedia
• Latest Microdata Published Draft
• Rich Snippets
• Schema.org getting started

Observation - generalization - theory - prediction

Today we are collectively devouring the massive quantities of data available about human and social behaviour, and applying machine learning and data mining techniques to find patterns in those data sets. Papers in this area are inevitably about interesting and even provocative correlations between different data sets, or previously unobserved statistical patterns in the data.

When I work on these data sets myself, or read these papers, I find such patterns initially fascinating, almost obsessively so. But somehow I feel unsatisfied at the end. I want more. Why are we seeing these patterns? Reading Stumpf and Porter’s paper about Critical Truths in Power Laws made me realize, the feeling of dissatisfaction stems from our training in the basic processes of science. We’re never going beyond the first step, observation, to propose generative theories about processes.

Computer science is a young field so it’s not that we’re just being lazy scientists. It’s that this is the equivalent of the times when people watched two objects roll down an incline and tried to understand what made them roll at different speeds. Making the jump from empirical observations to a theory of force and friction coefficients was a huge advancement. It involved meticulously refining and controlling for variables.

Today in computational social sciences we’re like kids in a candy store of endless correlations, running around consuming new data sets and fancy graphs. But correlation studies are just junkfood science without theoretical underpinnings. We need to go beyond making observations and studying correlations, to principled attempts at generalization, which in turn lead to theories about how human and social systems work, and predictions that others can actually test.

One concrete step we could take as data scientists working with information about human and social behaviour is to focus on experiments that systematically improve our ability to control for variables, as opposed to jumping to the correlations. It’s definitely fascinating to see graphs correlating GPS traces with power laws, downtown areas with mobility patterns, or graphs of interactions patterns at conferences. But without the ability to control for variables it is not reuseable science– no one else can take that and use it as input to derive a theory, nor can they test it with other data sets because there’s not enough information about what the input variables even were, never mind how to control for them.

Experiments that peel back our understanding systematically, even if it doesn’t immediately lead to a deep insight or new law about human behaviour, provide scaffolding that others can build on. At this early stage in the science of human systems, identifying fundamental dependencies are precisely the deep insights we need, and as we piece them together they become a true science, not just a bunch of quick fix correlation highs.

it’d be super easy to add some google analytics code to this and generalize the code for more file types and have a kind of tinyurl for personal files on your own server.

as it says in the comments, of course your server must be set up to host files at this url and be configured to interpret the url (without the .html extension) as the correct (html) mime-type.

If you are using Nginx, add a location directive to your server context with the following:

1
2
3
4

location / {
  types   { }
  default_type  text/html;
}

The code itself is as follows. Make sure to customize the user variables in CAPS.

And using it looks like this:

Notes

Has a lot of good references on scaling laws but less about form and structure per se.

If we can understand the natural tendency of these systems, THEN we can understand how to ply our hand to manipulate them.

For example if planning projects themselves take place with a power law distribution of effort…

Could look at purchasing habits from slice.com. Do the locations of people’s purchasing habits reflect a power law? How does online shopping play a role. Since that is effectively shopping AT home, does the set of physical places people (are willing to) go get skewed farther away?

We could also look at cityscapes - building heights.

Perhaps even look at aggregate city level data to see if economic indicators over many years rise and fall with increasing complexity of the cityscape?

Notes

The central limit theorem for random variables of ANY heavy tailed distribution converges on a power law!!

So observing heavy tail does not necessarily mean a power law

Similarly it means that it’s more important to understand the heavy tail behaviour.

Thus, without a clear theory of a generative mechanism, the simple observation of power-law-like behaviour is insufficient to claim that a fundamental process is at work, rather than insufficient data or even just failure to disaggregate.

Assessing Urban Complexity. Serge Salat, Loeiz Bourdic, and Caroline Nowacki. International Journal of Sustainable Building Technology and Urban Development / December 2010

fascinating paper… introduces and (loosely) discusses the characteristics of several metrics for measuring complexity and diversity in ubran environments. is at times totally incomprehensible, and doesn’t implement or test the proposed metrics, but thought provoking and with a few great references.

Metrics:

• Shannon index - entropy over the relative distribution of arbitrary categories of urban objects - eg. movie theatres, restaurants, and schools. This is sort of their baseline/strawman from which they launch into an exploration of alternative metrics. This metric is described as “hardly operational or easy to use” though it seems frankly a lot simpler than some of the ones they introduce!

• Simpson Index - captures diversity in counts across categories, and is useful when the goal is to find or show uniformity of distribution. apparently a common measure of ecosystem diversity, and also known as the “Herfindahl-Hirschmann” index. It is defined as the difference between 1 and the sum over the square proportions of each object. ie, if there are 3 different categories, the metric looks at the relative proportions of those categories present (in the urban environment). small proportions become very small (eg. 0.1^2), large proportions become less small (0.9^2). so by summing over the square of each proportion, the smaller categories are less represented than the big ones. the result is that distributions with higher variance (~ complexity) end up closer to 0. the metric is normalized so that the values lie between 0 and 1 no matter how many categories there are, and uniform distribution always result in a value of 1. I guess this is like a normalized variance measure.

• Simpson spatial index - the above Simpson index applied to surface area, so that each proportion now measures the area of a component, ni (say, schools, or parks) over the total area of a region.

• Salingaros Index - captures diversity of scale. Salingaros’ work suggests that the “multiplicity,” p, (number/relative frequency) of an element (say, roads) of size x, in a given category, should be inversely proportional to x^m, where m is the fractal dimension of the city. Since this relationship represents the ideal, the authors offer a derived metric, the Salingaros distance, that measures the distance of the observed distribution from this ideal. (ie, the desired power law behaviour versus the observed behaviour). the desired values is 0, ie, 0 distance.

The authors also introduce something they call the “structural distribution” and provide a measure of the distance of the Simson index from this structural distribution. but i honestly have no idea what they mean by “structural distribution.” hmm.

interesting to think about the factors involved in getting good answers from surveys, and how they relate to human sensor networks (HSNs), where human psychology may well play a role in the data gathered. cf. the three mentioned in the wikipedia article:

• central tendency bias
• acquiescence bias
• social desireability bias

questions:

• does the way we frame questions to human sensor networks affect the outcome?
• is the role of the human in the sensing, fundamentally inseparable from the measurement/sensing taking place?
• can we show if/how aggregation techniques can be used to normalize out human biases, or at least show a functional dependence between the convergence of HSN answers on the “correct” answer and, either, numbers (ie, more answers converges on the right answer); or, are there other parameters involved in that convergence?
• or could we even design human computation algorithms to take strategic advantage of relavent subjective psychological biases or tendencies, to improve convergence rates?

Subjective Summary: The authors build a system for crowdsourcing a task that involved so-called “global constraints,” ie, where the tasks is not just a simple sum of the individual inputs. The example in the paper is creating an itinerary. there is dynamic interaction between the individual tasks a user does (say, suggest an itinerary item, or demote a recommendation by another user) and the global output (a single itinerary). However, they didn’t really formalize the notion of global constraint or come up with heuristics or a formalism for understanding the interaction between the individual actions and the global outcome. it’s not clear to me how this is really different than crowdsourcing.

Questions/interests:

• formalize the notion of global constraints
• come up with a formalism for the relationship between individual actions and how they contribute to the global outcome.

The garden city design involved a radial graph structure. In Howard’s vision, there was a central hub in the middle with the largest population, and then smaller satellite settlements, with roads connecting neighbouring nodes in the graph. The space in between the nodes was park and farmland.

The original Garden Cities of Tomorrow book was published before cars, and before the explosive migration of humans to high density city centers. Today, one of the largest challenges associated with cities is transportation of people and goods. Public transportation faces the challenge that, towards the edges of the city, there is less frequent demand for places that are more spread out. A public transportation system could not efficiently meet the needs of all these locations. So, car cultures prevail in major cities, even though they create terrible congestion and have massive environmental costs.

Is it possible to create a city free from cars, that actually works? We can address the mismatch between supply and demand of public transportation if there is no sprawl. Perhaps we could revive Howard’s garden cities, with a modern twist. The roads between each node of the graph should be filled not with cars, but with dedicated, high frequency shuttles (SkyTran, anyone?).

Each node, or satellite, would be sized such that reaching any area within that node would be a short trip by foot and bike. Within each node, there would be no cars. Depending on the distance between nodes, shuttles could take some time. Which suggests another important vector for design: why build transportation as isolated functional systems, exclusively used for moving humans (or goods). Why should that system not also include restaurants? Or meeting spaces? Perhaps large windows and expansive views could provide a nice setting for the libraries of tomorrow?

I’m certainly not trained in urban planning or design, and among other things there’s a worthwhile debate about the role of planning, and the extent to which cities should (or even can) be planned. But if nothing else, transportation infrastructure sets in place basic rules that fundamentally influence the behaviour that takes place within cities. If we successfully hack the transportation layout, maybe we can achieve more efficient and functional settlements. How much are we contemplating radical alternative layouts and plans for human settlements, and the full scope of their implications for human dynamics and sustainability?

‘Grown-ups’ have various types of playgrounds, from parks and open space to cafes and bars.

I want to start a new type of playground for ideation and collaboration. A series of parks across the city with whiteboards, notepads, and prototyping material. If my friends and I were going to work on developing an idea, we could go here. Or you might go hang out there are participate in brainstorms with other people as they came by. Some people might leave behind ideas and others might extend them. It would be a literal creative commons.

You could imagine a mechanism for cities, governments, families, or individuals to deposit challenges that need brainstorming. If I have time after work I could go here and work on a problem that someone in my community is facing.

Having these spaces would help to create a culture of innovation, creativity and mutual assistance. One where such activities are valued by giving them space and time in our communities. It would support people in being proactive problem solvers, and looking towards the future rather than (only) being concerned with the present. It would say that we are all invited to actively create the world around us.

What if you had a journal of scientific ideas woven around fictional stories and characters? Where your job was to convince your readers of the validity of that story, and the technical details just happened to be plausible.

Two variations

1. Peer reviewers would get to decide the ending of the story. If they like your story, they could leave it as is. If they don’t, where a traditional journal would reject the paper, here the reviewers could modify the ending to have the main character, say, wake up from a dream and reflect on all the ways the idea didn’t make sense– or modify it in any other way the editors deem appropriate. stories could be continued by others or forked, creating multidimensional ancestral trees of ideas.

2. Different journals might each have their own fictional world in which every story (paper) takes place. The world represents the common starting assumptions of the journal editors. It might be a virtual world (a la second life or world of warcraft), or a conceptual world such as the meta world set up by Neal Stephenson and friends for their ongoing serial book the mongoliad. Scientists’ new ideas would be formulated such that they could be incorporated into these worlds– perhaps they are games– either as a software tool or a concept, and then tested by players or characters in that world. These worlds form a spectrum, from highly esoteric special-interest groups to, in the limit, deploying these ideas in our own “real” world.

This approach to science involves people as player-testers, extends the model of citizen science used in projects such as SETI at home and others. It is something they could have a reputation in, possibly even get paid for. new ideas and concepts would be subject to actual use and testing in these worlds, raising the bar of prototypes from throw-away code to alpha releases others can use (or extend).

There are interesting social hacks/incentives around publishing, too, if you have to let go of what the ending to your story might be. How confident are you that your characters or their ideas wouldn’t be killed off by the editors or other players? There is social capital at stake.

It also has the effect of coupling science more tightly with the nature of our social structures. Social structures are now re-cast as experimental environments for us to explore and see where things break.

Getting it to install

• hack the Rakefile just a wee bit: copy the existing ‘install’ task near the top of the file, to a new task called install_simple. install_simple is almost exactly the same as install, except that it only copies the theme’s source directory, without worrying about the sass directory (since in my case i’m not using sass). you could also make a conditional clause in the existing install task, or take some other approach. there’s plenty of ways to do it, this was just my first hack at getting it working.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

desc "Initial setup for simple Octopress theme: copies the custom theme into
the path of Jekyll's generator. run rake install_simple[theme_name]"
task :install_simple, :theme do |t, args|
  if File.directory?(source_dir) || File.directory?("css")
    abort("rake aborted!") if ask("A theme is already installed, proceeding will overwrite existing files. Are you sure?", ['y', 'n']) == 'n'
  end
  # copy theme into working Jekyll directories
  theme = args.theme
  puts "## Copying "+theme+" theme into ./#{source_dir} and ./css"
  mkdir_p source_dir
  cp_r "#{themes_dir}/#{theme}/source/.", source_dir
  mkdir_p "#{source_dir}/#{posts_dir}"
  mkdir_p public_dir
end

• mv plugins/category_generator.rb{,.ignore} (or otherwise get ride of it) because otherwise when you install the theme, jekyll will barf an error that there’s no category_index.html layout (which there isn’t since, in my case, i’m not using one). there’s certainly a better way to do this.
• mkdir -p .themes/my-theme/source
• put css files in .themes/my-theme/source/css (it’s essentially the root of your theme.)
• .themes/my-theme/source/_layouts are where octopress will look for your templates, eg. default.html, page.html
• .themes/my-theme/source/_includes are where octopress will look for any file you include in your _layout files, eg. say default.html uses an include statement like { % include head.html % }
• inside the theme dir create directories for images, javascript, etc. as needed.

Setting up a basic theme

you can obviously do whatever you want here, but i downloaded the html5 boilerplate as my starting point.

• customize boilterplate’s style.css and place it in the css/ directory
• take the index.html file given in boilerplate, and use it as the starting point for default.html. i found that i broke index.html down into the following files in the _include directory:
  • head.html, for the opening <html>, and <head> … </head>
  • header.html, for the site header
  • bottomjs.html, for your google analytics and jquery includes
  • footer.html, for your site footer
  • foot.html, for the closing html

  all this (except the <header>…</header> section) is already in boilterplate’s index.html, so just cut and paste it into the relevant files inside the _includes directory

• then create your _layouts/default.html file as:

1
2
3
4
5
6
7
8
9
10
11
12
13

{% capture root_url %} {{ site.root | strip_slash }} {% endcapture %}

{% include head.html %}

{% include header.html %}

{{ content | expand_urls: root_url }}

{% include bottomjs.html %}  

{% include footer.html %}

{% include foot.html %}

• now .themes/my-theme/index.html can basically look like this:

1
2
3
4
5

---
 layout: default
---

YOUR CONTENT HERE

• when you issue rake generate later, an index.html file will be generated in the public directory, populated by the various include files in the specified layout, and YOUR CONTENT HERE will be inserted where we used the {{ content }} tag. YOUR CONTENT HERE can be whatever you want– more html tags, liquid tags, just text, or a combination of all three.

• to install your theme issue rake install_simple['my-theme'] which will, as it notes, overwrite your existing theme files (so if you want them, back them up outside the directory somewhere).

Adding additional pages

• customize _layouts/page.html, using the same concepts we used to create default.html.
• you might have noticed the first line in the template above says {% capture root_url %} {{ site.root | strip_slash }} {% endcapture %}. for any template where you want to make use of the {% root_url %} variable, you need to include this.
• create a new page in the standard octopress way: rake new_page[my-new-page]
• edit the file
• issue rake generate and rake preview as normal.

Imagine the learning ecosystem as a graph, where learning challenges are nodes, and the edges between nodes represent the various ways these challenges can be combined. this is essentially a break down of the monolithic course concept into smaller, atomic components. by doing so, we increase the possible ways these components can be combined, leading to a more diverse ecosystem of learning opportunities.

in peer-based learning, the more traditional constraints on who can be a so-called ‘teacher’ are removed. since we are creating a graph reprsenting pathways to learning, these peers represent additional nodes in our graph. they also increase the combinatorial possibilities in the system of learning.

manifesting this kind of diverse, rich structure, does not happen via monolithic design (no matter how well intentioned). it happens via open standards and platforms: composable components that share interfaces, and can be combined in arbitrary and surprising ways. just to emphasize this, if we consider our graph to be completely unstructured (remember, the nodes in our graph are made up of learners and content), the number of possible subsets of those components is the powerset of all the nodes, or 2ⁿ, where n is the number of nodes. (never mind that for each subset, one can also enumerate all possible fully connected combinations of those components… but i digress). go ahead, calculate 2ⁿ where n is the number of courses currently available on P2PU. It’s clearly a simplification, but you get the point: a lot of possibilities. learning everywhere! hoorah.

now, traditional learning environments have clearly “scaled.” so what do we mean when we talk about scalability of peer-based learning? let me offer a working definition, that is: to reach and benefit arbitrarily many people. in traditional learning environments, the delivery mechanism has scaled, but the relevance and effectiveness have not. our giant learning graph of challenges and peers provides a notion of what it means for the experience of individually optimized learning to scale.

can heterogeneity be standardized?

us humans thrive on recognition and feedback. the compliment (actually precursor) to P2PU’s work on challenges, has been badges. simple, atomic symbols of recognition and accomplishment. one of the big questions around badges has been, are they standardized? if not, how can they be meaningful? and if so, are we not just becoming the very thing we sought to change?

unfortunately, the diversity of standardization doesn’t scale with the diversity of learning mechanisms or individuals. standardization scales by selecting a set of activities or pathways that are recognized and meaningful to a broad group of people. we’ve seen that the less dependencies we can institute between components in the graph, the greater the heterogeneity that can be achieved. but by their nature, standardized recognition systems provide inventive to follow well-trodden pathways. these pathways impose a structure that is otherwise not present in our free-form graph. structure reduces the combinatorial possibilities.

another draw of standardization is that it provides encapsulation. it’s a way of putting a wrapper around a body of effort, and evaluating it en mass. we often assume that people seek standardization because they desire a shared or common assessment. but through encapsulation, standardization also simplifies evaluation, and makes that evaluation portable. but how does one know that a given standardized mechanism has been applied correctly? that it actually is standardized? typically, this is through oversight and enforcement mechanisms. and those mechanisms have overhead. the benefit has to outweigh the cost for that overhead to be worth the effort. which, again, keeps the tension on that system to reduce diversity (reduce overhead).

these questions do more than highlight open problems to be solved. they lay bare a fundamental tension, between developing an arbitrarily heterogeneous ecosystem of learning, educational practices, goals, and opportunities; and our desire for meaningful recognition. if we provide infinitely many ways to recognize peoples’ efforts, the recognition loses its meaning. if we provide a few focused ones, we constrain what and how people can achieve.

many independent learner types don’t seek recognition or credentials for their learning efforts, and it’s easy to dismiss the need for credentials and recognition as valuable primarily in job-oriented environments. i think there’s a few reasons to consider the role of recognition beyond this:

• one goal of organizations like P2PU is to show that there are alternatives to “the system” for those who have not thrived in it (or perhaps who have, but who seek other ways). part of succeeding in this goal is providing a way to recognize their approach, whatever it may be.
• as humans, we thrive off of feedback and recognition.
• as the world, and its needs, become more diverse, so do the various skills and ideas we need to address those needs. we live in a post-industrial world where innovation and creativity are highly sought-after skills (at least in more economically prosperous countries). understanding how to design a system which systematically incentivizes (and, as a function of that, recognizes) disruptive ideas, is a broadly important goal. to give just one example, science funding agencies are constantly struggling with this question in terms of how to select grantees.

unfortunately, i don’t know the answer (sorry if you’ve slogged through this far in hopes I did). IMHO, such tensions do not get “solved,” but rather understood and leveraged. for P2PU’s approach to truly scale, i believe we need to put some serious brainpower behind this question: how can we recognize and reward arbitrarily diverse learning without constraining what that learning is or how one gets there? and how do we design these rewards and incentives without pre-supposing we have any clue what the outputs will look like?