nanshe.box.spams_sandbox module

The spams_sandbox module provides mechanisms for sandboxing SPAMS.

Overview

SPAMS sometimes seems to step on the interpreter. As a result, we provide a number of strategies to address this, by launching it in a separate process so that it hopefully does not mess up the main interpreter. We also try to keep the module space clean. This seems to help.

API

exception nanshe.box.spams_sandbox.SPAMSException[source]

Bases: exceptions.Exception

nanshe.box.spams_sandbox.call_multiprocessing_array_spams_trainDL(X, *args, **kwargs)[source]

Designed to start spams.trainDL in a separate process and handle result in an unnoticeably different way.

It is necessary to run SPAMS in a separate process as segmentation faults have been discovered in later parts of the Python code dependent on whether SPAMS has run or not. It is suspected that spams may interfere with the interpreter. Thus, it should be sandboxed (run in a different Python interpreter) so that it doesn’t damage what happens in this one.

This particular version uses a multiprocessing.Array to share memory to return the resulting dictionary.

Parameters:
  • X (numpy.matrix) – a Fortran order NumPy Matrix with the same name as used by spams.trainDL (so if someone tries to use it as a keyword argument…).
  • *args (list) – a list of position arguments to pass to spams.trainDL.
  • **kwargs (dict) – a dictionary of keyword arguments to pass to spams.trainDL.

Note

This is somewhat faster than using multiprocessing.Queue.

nanshe.box.spams_sandbox.call_multiprocessing_queue_spams_trainDL(*args, **kwargs)[source]

Designed to start spams.trainDL in a separate process and handle the result in an unnoticeably different way.

It is necessary to run SPAMS in a separate process as segmentation faults have been discovered in later parts of the Python code dependent on whether SPAMS has run or not. It is suspected that spams may interfere with the interpreter. Thus, it should be sandboxed (run in a different Python interpreter) so that it doesn’t damage what happens in this one.

This particular version uses a multiprocessing.Queue to return the resulting dictionary.

Parameters:
  • *args (list) – a list of position arguments to pass to spams.trainDL.
  • **kwargs (dict) – a dictionary of keyword arguments to pass to spams.trainDL.
Returns:

the dictionary found
Return type:

result(numpy.matrix)
nanshe.box.spams_sandbox.call_spams_trainDL(*args, **kwargs)[source]

Encapsulates call to spams.trainDL. Ensures copy of results occur just in case. Designed to be like the multiprocessing calls.

Parameters:
  • *args (list) – a list of position arguments to pass to spams.trainDL.
  • **kwargs (dict) – a dictionary of keyword arguments to pass to spams.trainDL.

Note

For legacy.

nanshe.box.spams_sandbox.run_multiprocessing_array_spams_trainDL(result_array_type, result_array, X_array_type, X_array, D_is_arg=False, D_array_type=None, D_array=None, *args, **kwargs)[source]

Designed to start spams.trainDL in a separate process and handle the result in an unnoticeably different way.

It is necessary to run SPAMS in a separate process as segmentation faults have been discovered in later parts of the Python code dependent on whether SPAMS has run or not. It is suspected that spams may interfere with the interpreter. Thus, it should be sandboxed (run in a different Python interpreter) so that it doesn’t damage what happens in this one.

This particular version uses a multiprocessing.Array to share memory to return the resulting dictionary.

Parameters:
  • result_array_type (numpy.ctypeslib.ndpointer) – Unused will drop. A pointer type with properties needed by result_array.
  • result_array (multiprocessing.RawArray) – shared memory array to store results in.
  • X_array_type (numpy.ctypeslib.ndpointer) – Unused will drop. a pointer type with properties needed by X_array.
  • X_array (numpy.ndarray) – currently uses numpy ndarray as input.
  • D_is_arg (bool) – Whether D either is an arg and/or should be an arg.
  • D_array_type (numpy.ctypeslib.ndpointer) – Unused will drop. a pointer type with properties needed by D_array.
  • D_array (numpy.ndarray) – currently uses numpy ndarray as the initial dictionary.
  • *args (list) – a list of position arguments to pass to spams.trainDL.
  • **kwargs (dict) – a dictionary of keyword arguments to pass to spams.trainDL.

Note

This is somewhat faster than using multiprocessing.Queue.

nanshe.box.spams_sandbox.run_multiprocessing_queue_spams_trainDL(out_queue, *args, **kwargs)[source]

Designed to run spams.trainDL in a separate process.

It is necessary to run SPAMS in a separate process as segmentation faults have been discovered in later parts of the Python code dependent on whether SPAMS has run or not. It is suspected that spams may interfere with the interpreter. Thus, it should be sandboxed (run in a different Python interpreter) so that it doesn’t damage what happens in this one.

This particular version uses a multiprocessing.Queue to return the resulting dictionary.

Parameters:
  • out_queue (multiprocessing.Queue) – what will take the returned dictionary from spams.trainDL.
  • *args (list) – a list of position arguments to pass to spams.trainDL.
  • **kwargs (dict) – a dictionary of keyword arguments to pass to spams.trainDL.