R image: update python, pangeo-notebook and rstudio-server

[emiliom]

Update these core packages to recent versions. First time we update these since 2022.

See #90 (comment)

This update is in support of the OHW-espanol event in a couple of weeks. ref oceanhackweek/Hub-Management#6

[emiliom]

@abkfenris the build_push check failed:

Traceback (most recent call last):
  File "/opt/conda/bin/conda", line 12, in <module>
    from conda.cli import main
ModuleNotFoundError: No module named 'conda'

I built the image locally (make r-build) without any problems. Could you help me debug this??

Thanks!

[abkfenris]

You'd think conda would be able to find conda...

[abkfenris]

Hmm, it's a pretty old miniconda image, so I wonder if there are somehow multiple versions of conda conflicting.

--- a/r/Dockerfile
+++ b/r/Dockerfile
@@ -1,5 +1,5 @@
 #syntax=docker/dockerfile:1.3
-FROM continuumio/miniconda3:4.12.0@sha256:58b1c7df8d69655ffec017ede784a075e3c2e9feff0fc50ef65300fc75aa45ae
+FROM continuumio/miniconda3:24.9.2-0@sha256:39b820af05f0f9bb986ca145a0d077bbd3bd866256277dd869fefb93bd03cd21

 ENV NB_USER jovyan

I tried changing a handful of things, but in my tinkering this morning I couldn't get Rstudio to launch from Jupyterlab by running make r-lab.

[emiliom]

Thanks. I should've tried to running make r-lab too, but forgot about it. I tried it now, and sure enough, I run into the same error as on the GH action (and same one you're seeing, presumably):

File "/opt/conda/bin/conda", line 12, in <module>
    from conda.cli import main
ModuleNotFoundError: No module named 'conda'

Hmm.

[emiliom]

For what it's worth, I followed the steps in https://hub.docker.com/r/continuumio/miniconda3 to pull the latest miniconda3 image and launch jupyter notebook server. In a new notebook, from conda.cli import main worked. It also worked on the python shell.

[emiliom]

I landed on a solution: update conda before doing conda install. After adding conda update -n root conda && \ right before conda install, I'm able to launch jupyter lab with make r-lab!

But on to the next problem: when I launch the RStudio kernel from jlab, I get a 500: Internal Server Error. Sigh.

And the build_push check failed with the same error as before. So the conda update fixed that immediate problem on my end, locally, but not on the CI. Hmm.

Actually, re-reading your message:

I tried changing a handful of things, but in my tinkering this morning I couldn't get Rstudio to launch from Jupyterlab by running make r-lab.

I now see that you were already at that stage, where you could launch jlab but not launch Rstudio from jlab. I thought what you meant was that you couldn't launch jlab. I'll keep digging.

[emiliom]

I updated the miniconda image to the version you listed, then explicitly added two library deb packages that were causing more subtle errors (including with nb_conda_kernels, I think): libarchive13 and libfmt-dev.

The CI tests are passing now! But Rstudio is still not launching successfully from jupyter lab. Rather than an immediate 500 error, as before, there's a wait of a few seconds, then the 500 error plus "The error was: could not start rstudio in time".

I added libarchive13 to try to address this error when launching jupyter lab with make r-lab:

conda-libmamba-solver (libarchive.so.20: cannot open shared object file: No such file or directory)

This error doesn't prevent jlab from launching and behaving normally (it seems). But I figured it's something I could address. Adding libarchive13 didn't fix it (I also tried libarchive-dev). I'm going to try one more strategy I just ran into to solve the libarchive.so error.

[abkfenris]

If nb_conda_kernels is causing issues, we should be able to get rid of it for this image. It shouldn't be necessary to run Rstudio. I think I tested with it removed, but still couldn't get anything other than a 500 trying to launch Rstudio.

[abkfenris]

I might explore the links off of here: https://discourse.jupyter.org/t/a-template-repository-for-running-latest-rstudio-r-on-mybinder-org-your-jupyterhub/12235

[emiliom]

I've pushed my last commits for the day. I've now eliminated all conda / libmamba errors. The last set of errors involved conflicts between libmamba and libarchive coming from conda-forge vs main channels.

There are no clear errors left in the run-time logs from make r-build and make r-lab, other than these warnings on make r-lab, which I assume are not critical.

[W 2024-11-14 23:25:32.326 ServerApp] No web browser found: Error('could not locate runnable browser').
...
[W 2024-11-14 23:26:33.071 LabApp] Could not determine jupyterlab build status without nodejs

BUT, RStudio still doesn't launch 😞. I still get this behavior: there's a wait of a few seconds, then the 500 error plus "The error was: could not start rstudio in time".

And the errors emitted when I click on the RStudio launcher in Jupyter Lab:

r-1  | [E 2024-11-14 23:26:54.725 ServerApp] {
r-1  |       "Host": "127.0.0.1:8080",
r-1  |       "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
r-1  |       "Referer": "http://127.0.0.1:8080/lab",
r-1  |       "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:132.0) Gecko/20100101 Firefox/132.0"
r-1  |     }
r-1  | [E 2024-11-14 23:26:54.725 ServerApp] 500 GET /rstudio/ ([email protected]) 15033.48ms referer=http://127.0.0.1:8080/lab

[emiliom]

Sigh, the build_push check failed; I was expecting it to work. But it looks like it's a temporary blip on the CI:

ERROR: failed to solve: rpc error: code = Unknown desc = write /opt/conda/lib/python3.12/site-packages/babel/locale-data/ta.dat: no space left on device
Error: buildx failed with: ERROR: failed to solve: rpc error: code = Unknown desc = write /opt/conda/lib/python3.12/site-packages/babel/locale-data/ta.dat: no space left on device

I left nb_conda_kernels on the conda env, and everything is fine.

I might explore the links off of here: https://discourse.jupyter.org/t/a-template-repository-for-running-latest-rstudio-r-on-mybinder-org-your-jupyterhub/12235

I saw that earlier, but I didn't look into it.