When: Thursday, January 28 from noon - 1pm
Where: 200C Warren Hall, 2195 Hearst St (see building access instructions on parent page).
Event format: The reading group is a brown bag lunch (bring your own) with a short <20 min talk followed by ~40 min group discussion.
Presenter: Jenn Stringer, Associate CIO, Academic Engagement and Director, Educational Technology Services
Facilitator: David Greenbaum, Research IT
Jenn Stringer, ETS
Oliver Heyer, ETS
Anne-Sophie De Baets, ETS
Nicholas Matthijs, ETS
Bill Allison, CTO
Lorena Barba, George Washington University - BIDS visiting scholar
Norm Cheng, OCIO-TPO
Mark Chiang, IST-EDW
Aaron Culich, Research IT
Harrison Dekker, Library Data Lab
Adam Fuchs, IST-Database Services
David Greenbaum, Research IT
Jon Hayes, IST-bConnected
Chris Hoffman, Research IT
Rick Jaffe, Research IT
John Lowe, Research IT
Steve Masover, Research IT
Scott Peterson, Doe Library
Aron Roberts, Research IT
Patrick Schmitz, Research IT
Walter Stokes, IST-Database Services
David Willson, OCIO-Licensing
(See slides – principal and supplementary; also From Bricks to Clicks: The Potential of Data and Analytics in Higher Education, a report by the UK's Higher Education Commission)
Oliver: Institutional data creates a profile; that's mashed up with activity data to create alerts, suggest learning paths, etc.
Nico: Retention is a big focus of current use (application) of learning analytics at Berkeley.
Jenn/Nico/Oliver: Engagement Index. Based on engagement in LMS. Quantifies (LMS-centered) participation in the course. Can be a large part of a student's grade.
Jenn: Predictive analytics are controversial, whether they might inappropriately influence advisors. Help or hindrance.
Jenn: Data is being collected on students. A lot of data. It's not just UCB, nor is it necessarily under UCB control. Ownership is an issue, value of ownership raises issues (and who derives benefit from that value).
Jenn: Piazza (no contract w/ UCB) collects lots of data and UCB can't have it except for lots of money ($40K annually); using collected data to profile students for companies recruiting for employees. Not authenticated except via an e-mail address. Not willing to negotiate w UCB as institution.
Jenn: Canvas has been a better partner, but required banding together with multiple institutions to leverage client-power.
Nico: Some vendors think they own data collected on Berkeley's (or other institution's) students. This can be problematic.
Liana: Consider student's ownership of data about them as well.
Jenn: Irvine has published an "Instructional Cloud Computing Tools & Services" scorecard. Not scalable for each campus to do this independently.
Nico: To be fair, institutions don't want to pay market value. So we've brought these vendor issues on ourselves, in some respects.
Jenn: Consider spend on admin systems vs. teaching and learning systems.
Nico: 6 billion "Learning Activities" for 2.5 years of data
Nico: A learning event could be a student logging in, navigating to a course, creating a post, etc.
Nico: A lot of "learning record stores" are oriented to corporate training. Missing aspects of concern to EDU, for example, giving student access to their own data, what it's being sued for and by whom, opt-out options -- things that corporate training folks don't care about at all.
Nico: Interest in using anonymized data as input to research in student behavior, education, etc.
Nico: UCB (ETS) building a Learning Record Store on Amazon Web Services. RDS, data encrypted at rest. (see architecture diagram in slides). POC done w/ 100s of millions of "statements" currently stored.
Jenn: Privacy dashboard in prototype.
Patrick: What does FERPA have to do with Learning Analytics?
Jenn: Varies by institution. There are hard questions that will have to be decided going forward.
(Multiple:) Physical activity (fitbit, self-reported) ==> LMS. Oral Roberts University (fitbit).
Steve: Thinking about relation between what learning objects or activities are modeled in conformity with a standard like Caliper and evaluations made by educators about whether a student is learning and how well -- what are some of the arguments about what these modeled activities imply in a 'real' sense, and how does a group like ETS manage selection of models if questions like this remain unsettled?
Jenn: Currently PoC only. But in general, that's not an ETS question. ETS can advise about technical questions, but the judgment questions will be settled elsewhere.
Oliver: Learning Analytics is at an early stage. Right now we're collecting, and doing so as a PoC, the decisions about what we'll do with this data [how to processm and what meaning it can generate to what level of confidence] are yet to be made by campus governance bodies beyond ETS.
Nico: Raw data will never be exposed to faculty, it's not useful. What'll be useful is processed data that takes multiple elements and generates them into useful information.
Discussion about retention; about whether students can get notes out to create a paper; about whether students can demand their data be expunged from the record(s).
David: Campus academic leaders particularly interested in this question?
Jenn: We're putting the questions on the table, and that's generating interest from quite a few campus leaders.
[additional post-meeting discussion followed...]