jupyterlite-pyodide-lock

Addons

PyodideLockAddon

A JupyterLite addon for patching pyodide-lock.json files.

class jupyterlite_pyodide_lock.addons.lock.PyodideLockAddon(**kwargs: Any)

Bases: BaseAddon

Patches a pyodide to include pyodide-kernel and custom packages.

Can handle PEP508 specs, wheels, and their dependencies.

Special pyodide-specific .zip packages are not supported.

Create a configurable given a config config.

Parameters

configConfig

If this is empty, default values are used. If config is a Config instance, it will be used to configure the instance.

parentConfigurable instance, optional

The parent Configurable instance of this object.

Notes

Subclasses of Configurable must call the __init__() method of Configurable before doing anything else and using super():

class MyConfigurable(Configurable):
    def __init__(self, config=None):
        super(MyConfigurable, self).__init__(config=config)
        # Then any other code you need to finish initialization.

This ensures that instances will be configured properly.

aliases: ClassVar = {'pyodide-lock-date-epoch': 'PyodideLockAddon.lock_date_epoch'}

property bootstrap_packages: list[str]

Get wheels for loadPackages.

bootstrap_wheels c.PyodideLockAddon.bootstrap_wheels = TypedTuple()

packages names from the lockfile (or .whl URLs or local paths) to load before the solve, such as a custom micropip

constraints c.PyodideLockAddon.constraints = TypedTuple()

PEP-508 specifications that constrain the bootstrap solve. Requires micropip >=0.9.0.

copy_wheel(wheel) → Generator[dict[str, Any], None, None]

Copy one wheel to {output_dir}.

Parameters:

wheel (Path)

Return type:

Generator[dict[str, Any], None, None]

enabled c.PyodideLockAddon.enabled = Bool(False)

whether ‘pyodide-lock’ integration is enabled

extra_preload_packages c.PyodideLockAddon.extra_preload_packages = TypedTuple()

extra packages to add to PyodideAddon.loadPyodideOptions.packages. These will be downloaded at kernel startup, and installed, but _not_ imported to sys.modules

property federated_wheel_dirs: list[Path]

The locations of wheels referenced by federated labextensions.

get_packages() → list[Path]

Find all file-based packages to install with micropip.

Return type:

list[Path]

lock(*, packages, specs, constraints, lockfile) → bool

Generate the lockfile.

Parameters:

• packages (list[Path])

• specs (list[str])

• constraints (list[str])

• lockfile (Path)

Return type:

bool

lock_date_epoch c.PyodideLockAddon.lock_date_epoch = CInt(0)

a UNIX epoch timestamp: packages modified after this time will be removed from proxied JSON responses

locker c.PyodideLockAddon.locker = Enum('BrowserLocker')

approach to use for running pyodide and solving the lock: these will have further configuration options under the same-named configurable

property locker_config: Any

A preview of the locker config.

property package_candidates: list[str]

Get all paths (or URLs) that might be (or contain) packages.

packages c.PyodideLockAddon.packages = TypedTuple()

URLs of packages, or local (folders of) compatible wheels for pyodide dependencies

post_build(manager) → Generator[dict[str, Any], None, None]

Collect all the packages and generate a pyodide-lock.json file.

This includes those provided by federated labextensions (such as jupyterlite-pyodide-kernel itself), copied during build:federated_extensions, which will be left in-place.

Parameters:

manager (LiteManager)

Return type:

Generator[dict[str, Any], None, None]

post_init(manager) → Generator[dict[str, Any], None, None]

Handle downloading of packages to the package cache.

Parameters:

manager (LiteManager)

Return type:

Generator[dict[str, Any], None, None]

pre_status(manager) → Generator[dict[str, Any], None, None]

Patch configuration of PyodideAddon if needed.

Parameters:

manager (LiteManager)

Return type:

Generator[dict[str, Any], None, None]

preload_packages c.PyodideLockAddon.preload_packages = TypedTuple()

pyodide_kernel dependencies to add to PyodideAddon.loadPyodideOptions.packages. These will be downloaded and installed, but _not_ imported to sys.modules

pyodide_cdn_url Unicode('https://cdn.jsdelivr.net/pyodide/v0.27.6/full')

the URL prefix for all packages not managed by pyodide-lock

pyodide_url Unicode('https://github.com/pyodide/pyodide/releases/download/0.27.6/pyodide-core-0.27.6.tar.bz2')

a URL, folder, or path to a pyodide distribution, if not configured in PyodideAddon.pyodide_url

resolve_one_file_requirement(path_or_url, cache_root) → Generator[dict[str, Any], None, None]

Download a wheel, and copy to the cache.

Parameters:

• path_or_url (str | Path)

• cache_root (Path)

Return type:

Generator[dict[str, Any], None, None]

specs c.PyodideLockAddon.specs = TypedTuple()

PEP-508 specifications for pyodide dependencies

status(manager) → Generator[dict[str, Any], None, None]

Report on the status of pyodide-lock.

Parameters:

manager (LiteManager)

Return type:

Generator[dict[str, Any], None, None]

property well_known_packages: Path

The location of .whl in the {lite_dir} to pick up.

jupyterlite_pyodide_lock.addons.lock.list_packages(package_dir) → list[Path]

Get all wheels we know how to handle in a directory.

Parameters:

package_dir (Path)

Return type:

list[Path]

PyodideLockOfflineAddon

A JupyterLite addon for resolving remote pyodide-lock.json.

class jupyterlite_pyodide_lock.addons.offline.PyodideLockOfflineAddon(**kwargs: Any)

Bases: BaseAddon

Rewrite pyodide-lock.json with locally-downloaded packages.

Create a configurable given a config config.

Parameters

configConfig

If this is empty, default values are used. If config is a Config instance, it will be used to configure the instance.

parentConfigurable instance, optional

The parent Configurable instance of this object.

Notes

Subclasses of Configurable must call the __init__() method of Configurable before doing anything else and using super():

class MyConfigurable(Configurable):
    def __init__(self, config=None):
        super(MyConfigurable, self).__init__(config=config)
        # Then any other code you need to finish initialization.

This ensures that instances will be configured properly.

property all_excludes: list[str]

Get all exclusion patterns.

property all_includes: list[str]

Get all inclusion patterns.

enabled c.PyodideLockOfflineAddon.enabled = Bool(False)

whether ‘pyodide-lock’ integration is enabled

excludes c.PyodideLockOfflineAddon.excludes = TypedTuple()

regular expressions to exclude from downloading

extra_excludes c.PyodideLockOfflineAddon.extra_excludes = TypedTuple()

more regular expressions to exclude from downloading

extra_includes c.PyodideLockOfflineAddon.extra_includes = TypedTuple()

more regular expressions for package names to download for offline usage

get_included_names(raw_packages) → tuple[set[str], set[str]]

Generate the lock.

Parameters:

raw_packages (dict[str, dict[str, Any]])

Return type:

tuple[set[str], set[str]]

get_pruned_packges(raw_packages, leaf_included, dep_included) → dict[str, dict[str, Any]]

Provide a copy of packages, potentially with pruning.

Parameters:

• raw_packages (dict[str, dict[str, Any]])

• leaf_included (set[str])

• dep_included (set[str])

Return type:

dict[str, dict[str, Any]]

includes c.PyodideLockOfflineAddon.includes = TypedTuple()

regular expressions for package names to download for offline usage

is_included(pkg_name, includes, excludes) → bool

Get the URL and filename if a package should be downloaded.

Parameters:

• pkg_name (str)

• includes (list[str])

• excludes (list[str])

Return type:

bool

property offline_lockfile: Path

A convenience property for a derived offline pyodide-lock output.

post_build(manager) → Generator[dict[str, Any], None, None]

Collect all the packages and generate a pyodide-lock.json file.

Parameters:

manager (LiteManager)

Return type:

Generator[dict[str, Any], None, None]

prune c.PyodideLockOfflineAddon.prune = Bool(False)

prune packages not requested to be offline

resolve_offline() → bool

Download and rewrite lockfile with selected packages and dependencies.

Return type:

bool

resolve_one_offline(pkg_name, out_dir, stem, packages) → None

Rewrite a single package’s info (if needed).

Parameters:

• pkg_name (str)

• out_dir (Path)

• stem (str)

• packages (dict[str, dict[str, Any]])

Return type:

None

status(manager) → Generator[dict[str, Any], None, None]

Report on the status of offline pyodide-lock.

Parameters:

manager (LiteManager)

Return type:

Generator[dict[str, Any], None, None]

Lockers

BrowserLocker

Solve pyodide-lock with the browser manged as a naive subprocess.

jupyterlite_pyodide_lock.lockers.browser.BROWSERS = {'chrome': {'headless': ['--headless=new'], 'launch': ['google-chrome', '--new-window'], 'private_mode': ['--incognito'], 'profile': ['--user-data-dir={PROFILE_DIR}']}, 'chromium': {'headless': ['--headless=new'], 'launch': ['chromium-browser', '--new-window'], 'private_mode': ['--incognito'], 'profile': ['--user-data-dir={PROFILE_DIR}']}, 'firefox': {'headless': ['--headless'], 'launch': ['firefox'], 'private_mode': ['--private-window'], 'profile': ['--new-instance', '--profile', '{PROFILE_DIR}']}}

browser CLI args, keyed by configurable

jupyterlite_pyodide_lock.lockers.browser.BROWSER_CHROMIUM_BASE = {'headless': ['--headless=new'], 'private_mode': ['--incognito'], 'profile': ['--user-data-dir={PROFILE_DIR}']}

chromium base args

class jupyterlite_pyodide_lock.lockers.browser.BrowserLocker(**kwargs) → None

Bases: TornadoLocker

Use a web server and browser subprocess to build a pyodide-lock.json.

See tornado.TornadoLocker for server details.

Parameters:

kwargs (Any)

Create a configurable given a config config.

Parameters

configConfig

If this is empty, default values are used. If config is a Config instance, it will be used to configure the instance.

parentConfigurable instance, optional

The parent Configurable instance of this object.

Notes

Subclasses of Configurable must call the __init__() method of Configurable before doing anything else and using super():

class MyConfigurable(Configurable):
    def __init__(self, config=None):
        super(MyConfigurable, self).__init__(config=config)
        # Then any other code you need to finish initialization.

This ensures that instances will be configured properly.

param kwargs:

type kwargs:

Any

browser c.BrowserLocker.browser = Unicode('')

an alias for a pre-configured browser

browser_argv c.BrowserLocker.browser_argv = TypedTuple()

the non-URL arguments for the browser process: if configured, ignore ‘browser’, ‘headless’, ‘private_mode’, ‘temp_profile’, and ‘profile’

browser_cli_arg(browser, trait_name) → list[str]

Find the CLI args for specific browser by trait name.

Parameters:

• browser (str)

• trait_name (str)

Return type:

list[str]

cleanup() → None

Clean up the browser process and profile directory.

Return type:

None

ensure_temp_profile(baseline=None) → str

Create a temporary browser profile.

Parameters:

baseline (Path | None, default: None)

Return type:

str

extra_browser_argv c.BrowserLocker.extra_browser_argv = TypedTuple()

additional non-URL arguments for the browser process.

extra_micropip_args c.BrowserLocker.extra_micropip_args = Dict()

options for ‘micropip.install’

async fetch() → None

Open the browser to the lock page, and wait for it to finish.

Return type:

None

headless c.BrowserLocker.headless = Bool(True)

run the browser in headless mode

host c.BrowserLocker.host = Unicode('127.0.0.1')

the host on which to bind

port c.BrowserLocker.port = Int(0)

the port on which to listen

private_mode c.BrowserLocker.private_mode = Bool(True)

run the browser in private mode

profile c.BrowserLocker.profile = Unicode(None)

run the browser with a copy of the given profile directory

protocol c.BrowserLocker.protocol = Unicode('http')

the protocol to serve

pyodide_cdn_url c.BrowserLocker.pyodide_cdn_url = Unicode('https://cdn.jsdelivr.net/pyodide/v0.27.6/full')

remote URL for the version of a full pyodide distribution

pypi_api_url c.BrowserLocker.pypi_api_url = Unicode('https://pypi.org/pypi')

remote URL for a Warehouse-compatible JSON API

temp_profile c.BrowserLocker.temp_profile = Bool(True)

run the browser with a temporary profile: clobbered by profile

timeout c.BrowserLocker.timeout = Int(0)

seconds to wait for a solve

tornado_settings c.BrowserLocker.tornado_settings = Dict()

override settings used by the tornado server

Locker Bases

BaseLocker

Base classes for jupyterlite-pyodide-lock implementations.

class jupyterlite_pyodide_lock.lockers._base.BaseLocker(**kwargs) → None

Bases: LoggingConfigurable

Common traits and methods for ‘pyodide-lock.json’ resolving strategies.

Parameters:

kwargs (Any)

Create a configurable given a config config.

Parameters

configConfig

If this is empty, default values are used. If config is a Config instance, it will be used to configure the instance.

parentConfigurable instance, optional

The parent Configurable instance of this object.

Notes

Subclasses of Configurable must call the __init__() method of Configurable before doing anything else and using super():

class MyConfigurable(Configurable):
    def __init__(self, config=None):
        super(MyConfigurable, self).__init__(config=config)
        # Then any other code you need to finish initialization.

This ensures that instances will be configured properly.

param kwargs:

type kwargs:

Any

constraints List()

extra_micropip_args c.BaseLocker.extra_micropip_args = Dict()

options for ‘micropip.install’

lockfile Instance()

micropip_args Dict()

packages List()

parent Instance()

pyodide_cdn_url c.BaseLocker.pyodide_cdn_url = Unicode('https://cdn.jsdelivr.net/pyodide/v0.27.6/full')

remote URL for the version of a full pyodide distribution

pypi_api_url c.BaseLocker.pypi_api_url = Unicode('https://pypi.org/pypi')

remote URL for a Warehouse-compatible JSON API

pythonhosted_cdn_url Unicode('https://files.pythonhosted.org')

remote URL for python packages (third-party not supported)

async resolve() → bool | None

Asynchronous solve.

An async locker should overload this.

Return type:

bool | None

async resolve_async() → bool | None

Asynchronous solve that handles timeout.

An async locker should _not_ overload this unless it has some other means of timing out.

Return type:

bool | None

resolve_sync() → bool | None

Provide a sync facade for doing async solves, called by PyodideLockAddon.

If a locker is entirely synchronous, it can overload this.

Return type:

bool | None

specs List()

timeout c.BaseLocker.timeout = Int(0)

seconds to wait for a solve

TornadoLocker

Host a tornado web application to solve``pyodide-lock.json`` .

jupyterlite_pyodide_lock.lockers.tornado.THandler

a type for tornado rules

alias of tuple[str, type, dict[str, Any]]

class jupyterlite_pyodide_lock.lockers.tornado.TornadoLocker(**kwargs) → None

Bases: BaseLocker

Start a web server and a browser (somehow) to build a pyodide-lock.json.

For an example strategy, see browser.BrowserLocker.

The server serves a number of mostly-static files, with a fallback to any files in the output_dir.

GET of the page the client loads:

• /lock.html

POST/GET of the initial baseline lockfile, to be updated with the lock solution:

• /pyodide-lock.json

POST of log messages:

• /log

GET of a Warehouse/pythonhosted CDN proxied to configured remote URLs:

• /_proxy/pypi

• /_proxy/pythonhosted

If an {output_dir}/static/pyodide distribution is found, these will also be proxied from the configured URL.

Parameters:

kwargs (Any)

Create a configurable given a config config.

Parameters

configConfig

If this is empty, default values are used. If config is a Config instance, it will be used to configure the instance.

parentConfigurable instance, optional

The parent Configurable instance of this object.

Notes

Subclasses of Configurable must call the __init__() method of Configurable before doing anything else and using super():

class MyConfigurable(Configurable):
    def __init__(self, config=None):
        super(MyConfigurable, self).__init__(config=config)
        # Then any other code you need to finish initialization.

This ensures that instances will be configured properly.

param kwargs:

type kwargs:

Any

property base_url: str

The effective base URL.

property cache_dir: Path

The location of cached files discovered during the solve.

cleanup() → None

Handle any cleanup tasks, as needed by specific implementations.

Return type:

None

collect() → dict[str, Path]

Copy all packages in the cached lockfile to output_dir, and fix lock.

Return type:

dict[str, Path]

collect_one_package(package) → dict[str, Path]

Find a package in the cache.

Parameters:

package (dict[str, Any])

Return type:

dict[str, Path]

extra_micropip_args c.TornadoLocker.extra_micropip_args = Dict()

options for ‘micropip.install’

async fetch() → None

Actually perform the solve.

Return type:

None

fix_lock(found) → None

Fill in missing metadata from the micropip.freeze output.

Parameters:

found (dict[str, Path])

Return type:

None

fix_one_package(root_posix, lock_dir, package, found_path) → None

Update a pyodide-lock URL for deployment.

Parameters:

• root_posix (str)

• lock_dir (Path)

• package (dict[str, Any])

• found_path (Path)

Return type:

None

host c.TornadoLocker.host = Unicode('127.0.0.1')

the host on which to bind

property load_pyodide_options: dict[str, Any]

Provide default loadPyodide options.

property lock_html_url: str

The as-served URL for the lock HTML page.

property lockfile_cache: Path

The location of the updated lockfile.

port c.TornadoLocker.port = Int(0)

the port on which to listen

preflight() → None

Prepare the cache.

The PyPI cache is removed before each build, as the JSON cache is invalidated by both references to the temporary files.pythonhosted.org proxy and a potential change to lock_date_epoch.

Return type:

None

protocol c.TornadoLocker.protocol = Unicode('http')

the protocol to serve

pyodide_cdn_url c.TornadoLocker.pyodide_cdn_url = Unicode('https://cdn.jsdelivr.net/pyodide/v0.27.6/full')

remote URL for the version of a full pyodide distribution

pypi_api_url c.TornadoLocker.pypi_api_url = Unicode('https://pypi.org/pypi')

remote URL for a Warehouse-compatible JSON API

async resolve() → bool | None

Launch a web application, then delegate to actually run the solve.

Return type:

bool | None

timeout c.TornadoLocker.timeout = Int(0)

seconds to wait for a solve

tornado_settings c.TornadoLocker.tornado_settings = Dict()

override settings used by the tornado server