🇬🇱 QGreenland &
🌨️ CryoCloud

An open GIS collaboration for education and research

In this presentation…

• A problem in our community: Technical training is hard!
• (Part of) the solution: Teaching on a JupyterHub to reduce distractions, improve accessibility, and enhance learning. A QGreenland workshop collaboration success story.
• How you can do it to: Check out CryoCloud!

In this presentation, we’re going to talk about a community problem: Technical training about Earth science data and GIS is difficult, and often requires a complex software environment for learners.

After gaining some experience this year, we feel that teaching on a JupyterHub is part of the solution. This enables learners and instructors to focus more of their time on learning and less of it on troubleshooting.

We think this could be a good option for your next workshop!

Matt Fisher^1,2 - Software Developer

Twila Moon^1,2 - Researcher

Alyse Thurber² - Education and Outreach

Tasha Snow³ - Researcher

Trey Stafford^1,2 - Software Developer

1: National Snow and Ice Data Center (NSIDC), University of Colorado Boulder

2: Cooperative Institute for Research in Environmental Sciences (CIRES), University of Colorado Boulder

3: Colorado School of Mines

I’m Matt Fisher, and this is the QGreenland and CryoCloud team that authored this talk.

• Software developer for the last 15 years, currently at NSIDC
• Over the last 4 years, Trey Stafford and I authored QGreenland, and
• This year, we took on a new role as Instructors, with Alyse Thurber, of the QGreenland Researcher Workshop. More on that in coming slides!

If anything in this talk goes by too quickly, there will be a QR code at the final slide linking to the webpage for this presentation where you can find a copy of the slides and more on the authors.

TODO: Consider omitting last names in speaker notes.

What is QGreenland?

• All-in-one, open-source, Greenland-focused GIS environment for offline and online use with QGIS
• Curated interdisciplinary data package for research, learning, decision-making, and collaboration.

https://qgreenland.org

An open-source, Greenland-focused, interdisciplinary data package and GIS environment for research, learning, and decision-making.

QGreenland is built for online or offline use, for example in remote field work.

Community education and outreach are important parts of our grant-funded activities, so we’ve run several workshops over the last 3 years.

A problem in our community

Technical training is hard!

Planning and administering technical training is hard.

We need to consider our topic carefully, and make sure it’s sized right for the time you have.

We need to build up knowledge in the correct order.

We need to design exercises that will help learners remember their new knowledge.

We need to figure out how breakout rooms in Zoom work even though we thought we knew.

On top of these things, we need to worry about how learners will set up technical requirements. Question like: For our workshop, do we need loaner laptops? Do we need a Mac expert and a Windows expert on the teaching team for technical support?

🖥️ Variance in user systems

• Hardware, operating system, and user comfort level

• User configuration
  • Installed programming languages & versions
  • Dependency managers & installed dependency locations

☝️ Takes time away from teaching, planning, and learning.

In this talk, we’ll zoom in on variance in user systems as a distraction from all the other important things we just talked about.

• I’m only comfortable with one operating system myself.

• Many learners’ user configurations, like mine, are customized.

• We all, learners and instructors, only have so much time and attention to give.

❤️ Accessibility

Hardware, (some) operating systems, and time are expensive.

Accessibility is our responsibility!

This is also an accessibility and inclusivity problem.

Hardware, operating systems, time, and expertise are expensive.

Not everyone has a laptop with 32GB of RAM, or the free time to set up dependencies for a workshop.

This is a barrier to participation! In our role as educators, we feel responsible to reduce that barrier.

QGreenland workshop

The QGreenland researcher workshop was a 3-day workshop that occurred earlier this year.

Both Trey and myself are primarily software developers and newer to our role as educators, so we learned a lot as we went, especially from Alyse Thurber with CIRES Education & Outreach.

QGreenland workshop objectives

📡 Run a virtual workshop

🧪 Follow FAIR open science principles

• Findable
• Accessible ⭐
• Interoperable
• Reusable ⭐

🦄 Minimize or eliminate distractions from system variance

We want to run a virtual workshop for a diverse audience of researchers.

Because we want to teach FAIR scientific data principles, we want our workshop to also embody those principles. We gave ourselves a star where we think we did particularly well.

• A: We provided a small stipend to support equitable participation. Anyone can learn from our materials on our website, or contribute to our materials on GitHub.
• R: We authored administrative documentation to support others re-running our workshop.

Our workshop requires users to be able to run QGreenland and install several other dependencies, including GDAL, and we wanted to avoid supporting all the possible pitfalls setting those up.

Extra (skip in main reading):

• F: Our workshop is available as a website and as a public Git repository. We’re doing outreach to share the materials and outcomes.
• I: Our source materials are in a common authoring format: Markdown

QGreenland workshop outcomes

👩‍🔬 25 learners from all career stages 🇩🇪 🇵🇱 🇮🇳 🇫🇷 🇨🇦 🇺🇸

💸 Reduced costs by >$4000 by collaborating with CryoCloud

🌲 Produced sustainable, re-usable materials

🪐 Shared lessons learned in Jupyter Blog post

25 learners from around the world, and from all career stages joined and completed our workshop. Our only technical prerequisites were Internet access and a browser.

EXTRA NOTE: Internet access is no small barrier to participation, but our format enabled international participation without requiring travel, which we felt was important.

We reduced the costs of the computing infrastructure from an expected $4500 in setup cost to only $75 in cloud costs for a 3-day workshop by collaborating with the CryoCloud community.

We produced open-source educational materials that we hope will be useful for a long time.

And we did outreach activities like a guest post on the Jupyter Blog to share what we learned.

The QR code on the last slide will take you to that post.

How we worked around system variance

CryoCloud JupyterHub to the rescue!

The CryoCloud project aims to transition the NASA research community to cloud and open source work patterns. Our participants did their work on CryoCloud’s JupyterHub instead of on their personal computers.

The CryoCloud JupyterHub is operated by International Interactive Computing Collaboration, a fiscally-sponsored non-profit.

🪐 JupyterHub?

📝 Jupyter Notebook: Interactive computing file format.

🖥️ JupyterLab: A comprehensive browser-based environment for using Jupyter Notebooks, and more.

⛅ JupyterHub: JupyterLabs managed in the cloud, on demand.

Hopefully I’m not repeating too much from previous talks. I’ll go through this quickly:

📝 Jupyter Notebook: Interactive computing file format. You can view and edit notebooks with various tools including the official Jupyter “notebook server” (jupyter notebook), VSCode, PyCharm, and Google Colab.

🖥️ JupyterLab: A comprehensive browser-based environment for Jupyter Notebooks, including desktop-like tools such as file browser, terminal, package managers, image previewers, and tabular data editors.

⛅ JupyterHub: A system for managing JupyterLabs in the cloud so users can create a JupyterLab on demand with no prior knowledge.

☁️ Desktop GIS in the cloud

QGIS logo

Now let’s look at a quick demo of the GIS experience we set up for our workshop.

• Everything you see here is going to happen in the browser.
• First, we’re navigating to CryoInTheCloud.com.
• We’ll navigate to their JupyterHub link and log in.
• We can configure our personal cloud server. I want Python, and 1 CPU.
• This loading part is in real-time. Some parts of this video are sped up, indicated with fast-forward icons.
• My JupyterLab is ready!
• Just fast forwarding through pointing out a few cool things JupyterLab can do, like previewing GeoJSON, a terminal…
• Let’s open the desktop.
• Navigate to and open the QGreenland QGIS project.
• Loading here is sped up to replicate the experience learners had in our workshop. This demo was recorded on less optimized and less expensive configuration.
• Let’s explore some stuff that looks visually interesting. I’m looking at the “Runoff” variable from the RACMO climate model.
• This looks cool, so I want to see satellite imagery of the area.
• I found a neat looking feature, so I decided to measure it, and it’s a bit over 4km across.
• You can use any other QGIS analysis or processing features with this setup. This is just to share how smooth and familiar the desktop experience feels in the cloud!

If you want technical detail on the infrastructure, you can check out our Jupyter Blog post and engage with CryoCloud and 2i2c on GitHub! QR code on the last slide.

IMPORTANT: When demo is done, click the fullscreen toggle button. DO NOT press escape or F11.

🫵 You can do this too!

We think you can also do this for your next workshop.

💸 Cost is still high, so…

🤝 Collaboration with an existing community was key!

🫵 And you can do this too.

• Sign up for CryoCloud
• See a list of 2i2c JupyterHubs

As you saw in a previous slide, setting up a cloud JupyterHub with all the features you need for your workshop can be expensive.

Advance

Because of this, collaborating with an existing community was essential if we wanted to pursue this strategy.

We were fortunate to have established connections with CryoCloud to help us get started, but many communities in this space are eager for collaborators. This arrangement saved our team lots of money, and we also provided value to CryoCloud: all future users of the hub will have ready access to QGIS and QGreenland, and the CryoCloud community grew because we onboarded our learners as new users.

Advance

If you’re looking to run a workshop, consider an existing community to collaborate with, like CryoCloud! Sign up by following the QR code at the end.

2i2c publishes a list of all the JupyterHubs they operate. That’s another great place to find collaborators.

Extra: If you’re looking to fund these type of activities, consider doing it through operating a JupyterHub community!

🙇 Thank you

Also at AGU:

• U24B-03: CryoCloud JupyterHub for NASA Cryosphere communities: Open science in the cloud as a process, not a product
• SH51A-04: NASA Openscapes approaches and stories of kinder, open science in the Cloud
• SH51A-06: Accelerating scientific discovery for NASA Cryosphere communities with the CryoCloud JupyterHub

Thank you!

There are some other presentations coming up that you might be interested listed on this slide. You’ll be able to find these by following the QR code at the next slide, too.

U24B-03 is coming up later today, check it out!

❓ Q&A

https://nsidc.github.io/agu2023-qgreenland-cryocloud/references.html

Contact

Matthew.J.Fisher@colorado.edu

qgreenland.info@gmail.com

Join CryoCloud and chat with us on GitHub Discussions!

I’m excited to talk with you and hear your questions!

If you follow the QR code, it will take you to the website for this presentation.

There, you’ll see a collection of links, including: join us on GitHub Discussions, get started with CryoCloud, download QGreenland, or learn more about the authors of this talk.

🎉 Bonus slides

👎 Distractions

👎 Distractions contribute to learners’ cognitive load and reduce learning effectiveness.

👎 Distractions consume teaching time.

👎 Distractions consume planning time.

TODO: Make text on slide more succinct, move wordy stuff into notes

Planning: How many helpers do we need? What prep-work do we need participants to do (installing software, etc.)? Do we need loaner hardware? This time could be better spent on curriculum.

🤝 A mutually-beneficial collaboration

💸 Reduced costs by collaborating!

👎 2-4 weeks + ~$4500 to set up

👍 2-3 days to setup + ~$75 cloud costs

✨ Contributed new functionality to the CryoCloud hub

👐 Open-source desktop GIS in the cloud!

🎉 11 Pull Requests merged; 9 issues opened

CryoCloud & QGreenland both benefitted from this arrangement.

• Reduced costs benefit our workshop participants by allowing us to focus more on teaching
  • Without: $4500 + 2-4 weeks setup
  • With: $0 + 2-3 days setup + $75 cloud operating expenses
• CryoCloud is an inclusive open source project. Running workshops helps expand CryoCloud’s (and other hubs!) capabilities:
  • We merged 11 PRs and opened 9 issues to help improve CryoCloud.
  • We collaborated with 2i2c and CryoCloud to enable QGIS, an open source desktop GIS application that competes with ArcGIS, to be used effectively in the cloud by a classroom-sized group, to do real data analysis. This involved…
    • MIME type fixes so QGIS project files open when double-clicked.
    • Performance improvements so QGIS projects could load as fast as they do on local compute.
    • Demo on upcoming slide.
  • Misc. improvements:
    • Upgrading important dependencies like Git
    • Fixing support for man pages
    • Adding utilities our users and instructors would need
    • Improving documentation
    • Improving CI/CD automations
    • Eliminating warnings that distract from learning

📈 Supported CryoCloud community growth & evaluation of impact

👐 Our workshop learners joined the CryoCloud community!

💡 Inspired other institutions to replicate our advancements

🪐 Jupyter Blog post

• Running workshops helps expand the CryoCloud community

• Running workshops helps the CryoCloud team gather data (e.g. surveys) to measure their impact.

• We published a Jupyter Blog post describing our advancements. There will be a QR code at the end.

  • As a result, 2i2c reported increased requests from their clients for the functionality we demonstrated in our blog post.
  • More JupyterHubs adopted QGIS desktop functionality.