Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Replacement of dict type for numba as parameters of a python function

Tags:

python

numba

In my code, there are a lot of parameters which are constant during the running. I defined a dict type variable to store them. But I find that numba cannot support dict.

What is a better way to solve this?

like image 824
WZhao Avatar asked Sep 01 '17 15:09

WZhao


People also ask

Do dictionaries work with Numba?

Numba does not fully support the Python dict because it is an untyped container that can have any Python types as members. To generate efficient machine code, Numba needs the keys and the values of the dictionary to have fixed types, declared in advance. To achieve this, Numba has a typed dictionary, numba.

Should I use Cython or Numba?

Both Cython and Numba speeds up Python code even small number of operations. More the number of operations more is the speed up. However, performance gain by Cython saturates at around 100-150 times of Python. On the other hand, speed up gain by Numba increases steadily with number of operations.

What is Nopython mode?

In nopython mode, the Numba compiler will generate code that does not access the Python C API. This mode produces the highest performance code, but requires that the native types of all values in the function can be inferred.

What is Numba in python used for?

Numba reads the Python bytecode for a decorated function and combines this with information about the types of the input arguments to the function. It analyzes and optimizes your code, and finally uses the LLVM compiler library to generate a machine code version of your function, tailored to your CPU capabilities.


1 Answers

Assuming you have a function like this and you are fine by accessing it as attribute instead of by subscript:

import numba as nb

@nb.njit
def func(config):
    return config.c

You could use a collections.namedtuple here (like @JoshAdel mentioned):

import numpy as np
from collections import namedtuple

conf = namedtuple('conf', ['a', 'b', 'c'])

func(conf(1, 2.0, np.array([1,2,3], dtype=np.int64)))
# array([1, 2, 3], dtype=int64)

Or a jitclass:

spec = [('a', nb.int64),
        ('b', nb.float64),
        ('c', nb.int64[:])]

@nb.jitclass(spec)
class Conf:
    def __init__(self, a, b, c):
        self.a = a
        self.b = b
        self.c = c

func(Conf(1, 2.0, np.array([1,2,3], dtype=np.int64)))
# array([1, 2, 3], dtype=int64)

These can't replace all functionalities of a dictionary but these allow to pass in "a lot of parameters" as one instance.

like image 64
MSeifert Avatar answered Nov 03 '22 18:11

MSeifert