CALIOPOLIS

Professor Leiter is conducting a study of law faculty scholarly productivity (methodology here). The goal (or one goal) is…

What if you replace the word "scholarly" with "teaching"?

If you have seen this video…

…then you can imagine how hard it is to measure teaching impact in the same way as Professor Leiter is measuring scholarly impact but perhaps there are some objective numbers that can be discovered and measured.

• number of classes taught
• number of students in those classes
• number of credit hours those classes are worth

These can be culled from the course catalog and a the class sizes by survey.

• casebooks published (first time)
• casebooks updated
• supplemental materials published

Theoretically, we could find this information out from the commercial publisher websites and matching up authors and co-authors with the publication dates to get an average per year.

Harder to get would be…

• adoption rates for casebooks
• sales figures for casebooks and supplemental materials
• whether supplemental materials are required, suggested or recommended.

Some of this could be culled from the course syllabi – many of which are online. There is a certain amount of "everyone knows" regarding what are the most popular casebooks in the major subject areas, but the long tail is harder to measure.

Having a casebook in the marketplace will generate a certain amountof traffic for the authors who must "teach the teachers" as well. Thisillustrates the point that teachers teach students and teachers teachteachers. Should we measure both? It would seem that teachingteachers has more downstream impact than teaching students. The placesthat teachers teach teachers include presentations given relating toteaching at …

• AALS
• Conferences
• Workshops

What about non-traditional supplemental materials?

• CALI lessons written
• Websites related to educational activities
• Blogs and podcasts with educational intent like Classcaster

I can certainly measure the first item and there are over 100 law professors who have written a CALI lesson at some time in the past, but the others would be much harder to track. A lot of web-based educational material is behind a password or inside a learning management system.

None of this speaks to quality, however and so we look back to Leiter’s study. He measures quality by citation. If someone else (either a court or another faculty member) has made a citation to your work, it is an indicator of impact and therefor some measure of the works value.

The nearest thing to citation in teaching is casebook/material adoption, but this would not capture the whole picture.

It would seem that potential law students would be very interested in this type of information, especially in the 150 law schools that are not going to end up in Professor Leiter’s Top 30 ranking of scholarly impact.

I ran across Lee Arnold’s video explaining the Bush Tax Cut some months ago and felt that it conveyed a complex topic with extreme clarity in a very short amount of time. I was gratified to see that Arnold has a series of videos on YouTube that you can access from here.

What is particularly interesting is his use of symbols that are apparently derived from the work of Howard Odum. Arnold calls it ecolanguage. Wikipedia tells me…

…Howard Thomas Odum (1924–2002), known as H.T. Odum or Tom Odum, was an American ecosystem ecologist and a professor at the University of Florida. He was collectively known as one of the "Fathers of Ecology" together with his brother Eugene…

I am fascinated by Arnold’s multimedia creations that short, powerful and enlightening in so little space and time.

I am convinced that there is an interesting project to develop a similar set of symbols – and also multimedia presentations – that could bring clarity to teaching the law. This is an idea I have been working on for a while that I call "Talking Flowcharts".

The first examples I saw of them in law were done by William Andersen from the University of Washington School of Law and they are embodied in his CALI lessons on Administrative Law.

In the original CALI lessons, Andersen included video and audio of him talking and explaining the charts as he walked students through the Administrative Procedure Act. Unfortunately, this was 1993 and the size of the media files was too large to be able to effectively distribute and we have never gone back to re-integrate them now that video and audio flies around the web with such ease.

James Maule, author of many, many CALI Tax lessons likes to point to the work of Arnold Mitchel, an International Tax Attorney who has created hundreds of charts explaining short concepts on his area of practice.

Finally, there is the excellent work of Professor Karl Manheim on his Constitutional Law charts.

All of these charts can be produced using Visio or SmartDraw or other tools and it’s would be somewhat straightforward to use Camtasia to create screencasts where narration could be added, but I my goal is larger. I want to create a consistent system or software that lets any law faculty easily create their own talking flowchart. Lee Arnold’s symbology makes me think that we could create something that would resonate with a large number of faculty and would generate multimedia presentations that are powerful and instructive.

I may be wrong however.

Law can be viciously complex and difficult to reduce to such symbols and the whole project would be open to the criticism of oversimplification. I would counter, however, that the talking flowcharts should not replace, but supplement – the usual argument for new teaching materials.

I would also contend that a talking flowchart is closer to what faculty do in the classroom with the chalk … er … whiteboard. They talk and draw circles and arrows and stab and gesture to emphasize key points. Talking flowcharts would have to imbue those same "gestures".

More exploration on this topic to come.

I have compiled the results from the 2007 Survey of Law Students who were in podcasted courses.

First, the number of students responding was a less than half from last year (120 in 2007 vs. 300 in 2006).

A couple of interesting trends are noticable. More students knew about podcasting this time around.

More students used portable MP3 players to listen to podcasts than before (24% vs. 17%), but the PC was the primary listening device.

More students listened to podcasts from other professors (15% vs. 8%), so awareness of podcasting professors is growing.

Podcasts as attendance-supressors seemed to decline with this survey. 2% said they attended less classes vs. 7% last year. 11% said they skipped classes vs. 12% last year.

Students rated podcasts value as EXCELLENT or ABOVE AVERAGE at about the same rate – 75% in 2007 vs. 74%.

The summary report is available here – 2007Survey.pdf – in PDF format.

The summary of comments is here – 2007SurveyComments.pdf

I visit law school websites all the time and I have come to the conclusion that they fall into the following categories…

• The "Where in the World" website where you cannot find a mailing address to save your life and thus cannot google-map or ship anything to anyone at the school. At best they have a PO box (which Fedex and UPS disdain) and makes me think that the entire school is just a PO drop box.
• The "I guess our university has a law school" website where all of the searches lead to the university library, the university faculty list and the university visitor information.
• The "Law Library Ascendant" website where the school has a home page and all the links lead to law library-branded pages.
• The "CNN Law School" website where there are dozens of press releases, faculty publication, event calendars and weather in <our fair city>. Must be run by the public affairs department.
  
• The "Jackson Pollock is our Webmaster" where I can’t tell where anything is going.

Almost all law school websites have at least the following on the home page…

• a picture of a brick building,
• Happy, happy, fun time, multi-cultural, not-stressed-out students fromwearing LL Bean clothes. You never see a picture of a real law students (gangsta jeans, nine-inch nails t-shirt, NPR baseball cap topsiders/no socks and starbucks cup firmly in hand).
  
• Menus…lots of menus

Which one is your website?

The Internet Bar Organization is sponsoring a contest for law students (and other qualified grads) where the prize if all expenses paid to Hong Kong to attend the International Online Dispute Resolution Group
Forum in Hong Kong.

Dan Rainey, Director of the Office of Alternative Dispute Resolution Services at the National Mediation Board says…

… and he adds …

The contest will run in three phases. The first is an online discussion open to all, wherein the contestants will engage in an online dialogue regarding trusted online communities. After the first round of discussions, the judges will select a smaller number of contestants to continue the discussion in a more focused manner, and then in early July the judges will pick up to 15 contestants to write a paper about the discussion and their notions of how to create a trusted online community. From the submitted papers, the judges will pick one as the grand prize winner, and that person will become the recorder for the

December meeting of the International Online Dispute Resolution Group Forum in Hong Kong. All expenses to the conference will be paid for the
grand prize winner.

It looks like an interesting competition and I will be watching the developments of the Internet Bar in the future.

The Free Software Foundation is doing research (look under April 12, 2007 New Flash) on when the term "Intellectual Property" first started to be used and as part of that research they are asking folks to send them copies of the pages in old (1970’s-80’s) course catalogs where the term was first used in a course name or description.

Help them out.

Gene Koo of the Harvard Berkman Center has published a white paper titled "New Skills, New Learning: Legal Education and the Promise of Technology". The research was sponsored by LexisNexis and the results are both insightful and cogent.

I had several conversations with Gene about the project and he did a marvelous job pulling together the survey results and culling the information into a very useful report.

The paper is available as here as a wiki where you can contribute your own ideas and reactions (you can also get the PDF version from there as well).

Gene will also be giving a presentation on the paper at Berkman…

Tuesday, May 22
12:30pm – 1:30pm
23 Everett Street, Cambridge MA

The presentation will also be streamed live over the Web and in Second Life. More info here …

http://cyber.law.harvard.edu/home/webcast

Social Software or Web 2.0 are buzz phrases much overused and ill-defined these days, but I ran across an explanation by Gene Smith that helps to evaluate social software/web 2.0 websites.

Here are Gene’s definitions…

• Identity – a way of uniquely identifying people in the system
• Presence – a way of knowing who is online, available or otherwise nearby
• Relationships – a way of describing how two users in the system are related (e.g. in Flickr, people can be contacts, friends of family)
• Conversations – a way of talking to other people through the system
• Groups – a way of forming communities of interest
• Reputation – a way of knowing the status of other people in the system (who’s a good citizen? who can be trusted?)
• Sharing – a way of sharing things that are meaningful to participants (like photos or videos)

Gene created the honeycomb that is displayed at the top this post. I want to use this to explore this in relation to websites and services used by law faculty and projects that CALI is working on right now.

First, let’s look at SSRN.

• Identity – the authors are known,
• Presence – there is no way to know who downloads your papers or who is currently online at any time,
• Relationships – no way to establish relationships between papers or individuals,
• Reputations – the download counts are an oft-discussed proxy for reputation,
• Groups – there is a sense of groups by the institutions that establish collections of papers, but this is not controllable or customizable by visitors to the website,
• Conversations – conversations about the papers are held elsewhere (i.e. blogs, watercoolers, etc.)
• Sharing – SSRN is all about sharing your scholarly papers.

Now let’s look at Blogs and by this I mean blogs where law professors hang out like the Law Professor’s Blog Network, Volokh Conspiracy, PrawfsBlog, etc.

• Identity – the authors are known,
• Presence – no way to know who is online in real time, though sometimes the comment stream can seem almost real time,
• Relationships – no way to capture the relationships made,
• Reputation – this is hard to measure. Some law faculty certainly have changed their reputation in the community via their blogs, but there is generally no software metric for this except perhaps visitor or hit counts, but this is true for any website, so I come down on blogs having no reputation support built-in,
• Groups – All of the blogs mentioned above are "group blogs", but there is no way for visitors to join the group,
• Conversations – blogs are all about conversations,
• Sharing – the sharing is in the conversation and not in file sharing really.

Now let’s look at Classcaster which is a blog network where law faculty mostly post their course podcasts (though it can be used for any legal educationl purpose – and is).

• Identity – The podcasters are known,
• Presence – no presence,
• Relationships – no relationship management,
• Reputation – no reputation metrics,
• Groups – no groups except the course for which the podcasts are made,
• Conversation – Classcaster hosts blogs so conversation is possible, but this has only been lightly used. The conversations about the course take place elsewhere,
• Sharing – the podcasts are shared (sometimes with anyone on the web).

Finally, let’s look at TWEN which is the The West Education Network and a service offered by Thomson/West for law faculty to create course websites.

• Identity – everyone is known,
• Presence – I don’t believe that TWEN has the feature of displaying currently logged in users – I could be wrong,
• Relationships – no way to capture relationships,
• Reputation – no reputation system that I know of,
• Groups – instructors create groups that are their courses, but there is no way for users to create ad-hoc groups,
• Conversations – TWEN has extensive capabilities for conversation – threaded discussions, comments, etc.
• Sharing – TWEN sites allow faculty to share files, links, documents with their students. I am not sure that students can share things with each other though, so perhaps this should be light green.

This interesting. It’s important to note that not every social website needs to hit every point in the honeycomb to be useful or successful, it’s just a way to understand what social software/web 2.0 means.

It does make me think about what it would take to add features or services to make blogs more reputation-aware or for SSRN to capture the conversations about scholarship.

CALI is working on a series of projects that address almost all of these ideas – but not in a single website – rather, a constellation of websites. Here’s the honeycomb with the names of the CALI projects filled in…

• Identity – with over 100,000 law students and law faculty registered at the CALI website, this is where our identity system is centered,
• Presence – so far, nothing we are planning has presence built-in, but we may consider adding this in the future,
• Relationships – In eLangdell, users will form "relationships" by virtue of their adoption of materials as course materials. There is a natural relationship between casebook authors and the faculty who adopt the casebook. This is true at every level of adoption of a teaching resource. It is less clear whether any of our projects make these relationships more explicit.
• Reputation – ScholarshipPulse, The Legal Education Commons andeLangdell have reputation elements designed into them. InScholarshipPulse, the idea is to allow visitors to filter commentsbased on the status of the person who made the comment. In the LegalEducation Commons, users will rate materials by their suitability totask (and perhaps other metrics) so that these "signals" can be seen byothers.
• Groups – CALIGroups is intended to explicitly support "communites of practice" of law faculty.
• Conversations – ScholarshipPulse is all about conversations about scholarship. The Legal Education Commons and eLangdell will support conversations about teaching materials. CALIGroups will support conversations within a community.
• Sharing – Legal Education Commons is all about sharing. eLangdell will support sharing between law faculty using each others’ teaching materials. CALISpaces is for students to share materials with their instructors and classemates and CALIGroups will support sharing within a community of teachers.

This little exercise illustrates the extent of the projects we are working on and how they map to the social software/web 2.0 space. They also give me insight into how they fit together and hopefully will make it easier for me to explain these projects to others as they come online.

The theme for this year’s Conference for Law School Computing …

Legal Education and IT:Mirage or Oasis?

I fear that this choice of theme may be mis-interpreted, so let me provide some thoughts.

As conference themes go, it’s a little ambiguous – which is good. By leaving wide latitude for interpretation, the speakers can riff of it and it’s intended to be a bit challenging and thought-provoking.

This article "Who Needs a CIO" by Chris Anderson (author of the Long Tail book) articulates it better than I can…

http://www.longtail.com/the_long_tail/2007/02/who_needs_a_cio.html

Like Anderson, I fear that IT in legal education is becoming irrelevant either through outsourcing, technology-shifting to the ends of the network, malaise or lack of vision. I never really subscribed to the "technology for technology’s sake" approach and what with the law school’s traditional resistance to change, it’s hard to introduce new tools, ideas or services that make things more efficient or help law schools achieve their purpose.

This is the mirage. IT slowly disappears into the email that can be had from Yahoo, the research that can be had from Google and so on.

There is another angle to the theme. IT is becoming so pervasive or "built-in" that IT thinking must shift it’s focus away from plumbing and closer to the actual goals of legal education. This new type of thinking affects priorities, job skills and decision-making all over the place. You can no longer be simply in charge of the computers or in charge of the lab or in charge of the network – you have to be a collaborator and coordinator of services to the people that use the computers, labs and network. This is the oasis – it’s just there, it just works, it nourishes existing activities instead of impeding them.

Finally, there is the strategic. This too is part of the oasis. It’s a somewhat distant, but reachable promise that IT can make things possible that were not possible without it. I really am astounded by the power of the tools we have available today, but I think we are only just beginning to figure out how to use them effectively.

As we all know, IT has not always been equal to its hype. Where is our natural language search? AI? … and what about the problems it has created? Spam! Viri! Phishing! Information Overload!

This too is the mirage – a promise not kept or a hope not realized.

There are plenty of excellent sessions at this year’s conference.

Here’s the conference home page where you can register and see the preliminary and constantly changing agenda…

http://www.cali.org/conference

Hope to see you there.

An interesting article at the SocialText wiki blog got to me to thinking about the post-blog/wiki-web, when these blogs and wikis and tags and RSS feeds are not artifacts in themselves, but are embedded into …. everything else.

The two common themes I find to measure all applications of the read/write web … by which I mean …

• blogs
• wikis
• rss feeds
• IM
• email
• comments
• tags and taggregators like del.icio.us

… are TIME and PERMANENCE.

TIME

Blogs, RSS feeds, comments and some email are highly TIME dependent. They lose their relevance over time. This is not an absolute of course, but you get the drift.

Wikis can either be somewhat independent of time (like Wikipedia which is capturing knowledge for all the future) or highly relevant for a period of time (the life of a project).

Tags can be both transitory and permanent in aggregation.

IM is rightnow and email is less so (though some people feel it is 24-48 hours and then fades fast).

PERMANENCE

Everything on the list except email and IMs are permanent (IMs and email are only permanent depending on document retention policies and subpeonas), but that’s not my point. I mean permanence in terms of intention. IMs and email are not intended to be permanent, except in the most archived for posterity sort of way.

Blogs posts are intended to be a permanent record of work or thinking and their comments less so.

Wikis are intended to create permanent, though malleable artifacts – captured knowledge.

RSS feeds are as permanent as the blogger is alive – a kind of heartbeat or measure of ‘aliveness’.

I subscribe to a many blogs and RSS feeds, but if it’s dishing up too many posts-per-day, it feels like a noisesome child who won’t shut up and I throttle it back or remove it from my feed reader.

What does this all have to do with education?

Education is a kind of transitory thing that we do to ourselves with the intention of permanent change. We learn something and then we hope we don’t have to learn it again. We add to our permanent experience. We also depend on the permanence of the web (and Google’s good graces) to permanently store things so that we can learn about them (again and again) in the future. Google is the largeest just-in-time educational system out there.

Our lives used to be front-loaded with education – 12 or 16 years of it if you finished college. More for the Masters or PhD. Now it’s life-long learning (but it always was in many ways). I have considered going back to school for my PhD or for another Masters degree many times, but I always decide that I don’t want the degree, I want the knowledge and I can get that from books, blogs and Google. My job practically requires that I be in a constant state of learning and I suspect that is true for many of today’s knowledge workers.

I think the crude tools we use today for collaboration and commentary like blogs and wikis, will become less like end-point artifacts and more like features in every website and the wider worlds. Every store, product, person, concept, candidate and company will have multiple places where transitory (blogs, comments) and permanent (wikis) information can be found and contributed to.

Time is not well represented on the web. Everything almost always looks new. It would be cool if the software made the web page look more grey or well-thumbed so that we could immediately tell if we were reading old information. This is problematic, because ‘old’ is relative to the context. The web page that tells me if my flight is delayed is old to the point of uselessness the next day – unless I am studying yearly trends in flight delays, then the permanence of that information is valuable.

I am not sure of context rules, however.

Time and Permanence seem like good meta-identifiers of information and may help in understanding the larger semiotics of the webs.