Jetstream is a cloud computing system that became available to researchers in 2016 through the National Science Foundation XSEDE program. CyVerse Atmosphere is an open-source interface for Jetstream that allows researchers to quickly and easily provision their own cloud resources at no cost.
Our presenters will share their roadmaps for Jetstream and CyVerse, and we’ll have open discussion about opportunities to collaborate and integrate emerging technologies for teaching and research such as JupyterHub and BinderHub.
When: Thursday, 2 November from 12 - 1pm
Where: 200C Warren Hall, 2195 Hearst St (see building access instructions on parent page).
What: Jetstream/CyVerse Roadmaps
Prior to the meeting, please review the following:
An overview of CyVerse (fka The iPlant Collaborative): https://doi.org/10.1371/journal.pbio.1002342
Bringing your tools to CyVerse Discovery Environment using Docker: https://f1000research.com/articles/5-1442/v3
An overview of Jetstream: https://dl.acm.org/citation.cfm?id=2792774
Because CyVerse Atmosphere is open-source, other projects are welcome to use Atmosphere directly or easily extend it for their own needs, given Atmosphere’s plugin architecture. Massachusetts Open Cloud is one such example: https://www.openstack.org/videos/boston-2017/moc-simple-gui-an-and-to-openstack-horizon
Edwin Skidmore, CyVerse
Jeremy Fischer, Jetstream
George Turner, Jetstream
Mike Lowe, Jetstream
Sanjana Sudarshan, Jetstream
Andrew Wiedlea, LBNL
Aaron Culich, Research IT
Barbara Gilson, SAIT (emeritus)
Jason Christopher, Research IT
Maurice Manning, Research IT
Morteza Faraji, Psychology
Nico Tripcevich, Archaeological Research Facility
Patrick Schmitz, Research IT
Quinn Dombrowski, Research IT
Rick Jaffe, Research IT
Scott Peterson, Library
Shaun O'Grady, Psychology
Steve Bird, Jetstream SysAdmin, IU
Steve Masover, Research IT
--> George Turner: Jetstream overview slides (PDF)
--> Edwin Skidmore: Atmosphere Roadmap, 2017-18 slides (PDF). Aimed at scientists who don't want to deal with the nuts and bolts of using cloud resources. Roadmap Q3 2018 -- Docker integration (containers); also ability for users to execute tools on a VM non-interactively (programmatically).
Edwin: CyVerse -- Dynamic tool installation on running instances of Atmosphere-managed VMs via UI -- pre-templated.
Steve: What tools? Who writes templates?
Edwin: Site operators. Common tools will be made available as examples and PoCs -- e.g., Jupyter, JupyterHub, R Studio, maybe others from CyVerse. Ansible-based, primarily.
George: Jetstream is very heavily leveraged in what Atmosphere is doing. Open to suggestion re: which packages to get installed into OpenStack. Docker and containers are very popular; Hadoop is installed but no one seems to be playing with it yet.
Maurice: How many taking advantage of OpenStack API. What use cases?
Jetstream Team: Surprised at early adoption. Most of the people use Atmosphere, but most of the cycles are used via API -- via gateways -- Galaxy users (~8000 of them).
Patrick: Kubernetes on roadmap? To, for example, provision a bunch of Jupyter instances.
Aaron: Packaging Kubernetes scripts to run in different cloud environments. Yuvi Panda's work in DSEP context at Berkeley.
Mike: Stuck in licensing questions/issues at the moment. Deferring technical work until licensing is worked out. Therefore it's tentative in the roadmap.
Shaun: MatLab image on Jetstream. Are versions up-to-date? Or version of MatLab current when image was created?
Jeremy: The latter. Looking at updating this.
Mike: We have a MatLab site license for Jetstream.
Shaun: Use case - Mechanical Turk - can a MT person be forwarded to an Atmosphere instance to perform computation? Can this be done easily?
Jeremy: If you have a science gateway that takes input. So the work is put back on you. No outbound restrictions, as long as what you want to talk to will talk to you...
Aaron/Nico/Mike/George: discussion about R Shiny. Nico has run it, but wasn't able to see the browser page that Shiny should have been rendering.
Shaun: Uploaded packages, data to images that are later shelved -- do the uploaded data persist?
George: Yes. It's image creation lite.