Error message: h5py.h5py_warnings.H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead

Question

I'm tring to run mbin for methylation analysis. But get error message: h5py.h5py_warnings.H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead. for several attempts, while trying to extract control IPDs with buildcontrols.
Environment:

• mbin version: 1.1.1
• Python version:2.7.12
• Operating System: centOS
• running under virtualenv

I thought it was, again, caused by version. What I've tried: I tried on a server with both python 3 and 2. And specified virtualenv to use python 2 as interpreter. mbin was successfully installed, but buidlcontrols didn't.

(virtualenv_for_mbin) [yx@localhost virtualenv_for_mbin]$ buildcontrols -i --procs=4 --control_pkl_name=control_means_190201.pkl /data1/yuxiao/userdata1/jobs/016/016504/data/aligned_reads.cmp.h5
2019-02-01 23:36:31 [INFO] Initiating dictionary of all possible motifs...
2019-02-01 23:36:31 [INFO]   - Adding 256 4-mer motifs...
2019-02-01 23:36:31 [INFO] Done: 256 possible contiguous motifs

2019-02-01 23:36:31 [INFO]   - Adding 1024 5-mer motifs...
2019-02-01 23:36:31 [INFO] Done: 1536 possible contiguous motifs

2019-02-01 23:36:31 [INFO]   - Adding 4096 6-mer motifs...
2019-02-01 23:36:32 [INFO] Done: 7680 possible contiguous motifs

2019-02-01 23:36:32 [INFO]   - Adding bipartite motifs to search space...
2019-02-01 23:36:33 [INFO] Done: 194560 possible bipartite motifs

2019-02-01 23:36:33 [INFO] 
2019-02-01 23:36:33 [INFO] Preparing to create new control data in ctrl_tmp
Traceback (most recent call last):
  File "/data2/Software/virtualenv_for_mbin/bin/buildcontrols", line 10, in <module>
    sys.exit(launch())
  File "/data2/Software/virtualenv_for_mbin/lib/python2.7/site-packages/mbin/controls.py", line 20, in launch
    extract_controls(opts, control_aln_fn)
  File "/data2/Software/virtualenv_for_mbin/lib/python2.7/site-packages/mbin/controls.py", line 40, in extract_controls
    opts           = controls.scan_WGA_aligns()
  File "/data2/Software/virtualenv_for_mbin/lib/python2.7/site-packages/mbin/controls.py", line 352, in scan_WGA_aligns
    reader = openIndexedAlignmentFile(self.control_aln_fn)
  File "/data2/Software/virtualenv_for_mbin/lib/python2.7/site-packages/pbcore/io/opener.py", line 52, in openIndexedAlignmentFile
    return CmpH5Reader(fname, sharedIndex=sharedIndex)
  File "/data2/Software/virtualenv_for_mbin/lib/python2.7/site-packages/pbcore/io/align/CmpH5IO.py", line 729, in __init__
    self._loadAlignmentInfo(sharedIndex)
  File "/data2/Software/virtualenv_for_mbin/lib/python2.7/site-packages/pbcore/io/align/CmpH5IO.py", line 745, in _loadAlignmentInfo
    rawAlignmentIndex = self.file["/AlnInfo/AlnIndex"].value
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/data2/Software/virtualenv_for_mbin/lib/python2.7/site-packages/h5py/_hl/dataset.py", line 313, in value
    "Use dataset[()] instead.", H5pyDeprecationWarning)
h5py.h5py_warnings.H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead.

I checked python version:

(virtualenv_for_mbin) [yx@localhost virtualenv_for_mbin]$ python -V
Python 3.6.5 :: Anaconda, Inc.

So, I immediately tried to run a new virtualenv.

[yx@localhost Software]$ mkdir venv_tmp
[yx@localhost Software]$ cd venv_tmp/
[yx@localhost venv_tmp]$ virtualenv --python=/usr/local/bin/python2.7 .
Running virtualenv with interpreter /usr/local/bin/python2.7
New python executable in /data2/Software/venv_tmp/bin/python2.7
Also creating executable in /data2/Software/venv_tmp/bin/python
Installing setuptools, pip, wheel...
done.
[yx@localhost venv_tmp]$ . bin/activate
(venv_tmp) [yx@localhost venv_tmp]$ python -V
Python 3.6.5 :: Anaconda, Inc.

Then, I turned to another server.....with only python 2.7 installed.

(venv_for_mbin) [yuxiao@localhost data_tmp]$ buildcontrols -i --procs=4 --control_pkl_name=control_means_190202.pkl aligned_reads.cmp.h5
2019-02-02 11:06:16 [INFO] Initiating dictionary of all possible motifs...
2019-02-02 11:06:16 [INFO]   - Adding 256 4-mer motifs...
2019-02-02 11:06:16 [INFO] Done: 256 possible contiguous motifs

2019-02-02 11:06:16 [INFO]   - Adding 1024 5-mer motifs...
2019-02-02 11:06:16 [INFO] Done: 1536 possible contiguous motifs

2019-02-02 11:06:16 [INFO]   - Adding 4096 6-mer motifs...
2019-02-02 11:06:16 [INFO] Done: 7680 possible contiguous motifs

2019-02-02 11:06:16 [INFO]   - Adding bipartite motifs to search space...
2019-02-02 11:06:18 [INFO] Done: 194560 possible bipartite motifs

2019-02-02 11:06:18 [INFO] 
2019-02-02 11:06:18 [INFO] Preparing to create new control data in ctrl_tmp
Traceback (most recent call last):
  File "/data1/yuxiao/venv_for_mbin/bin/buildcontrols", line 10, in <module>
    sys.exit(launch())
  File "/data1/yuxiao/venv_for_mbin/lib/python2.7/site-packages/mbin/controls.py", line 20, in launch
    extract_controls(opts, control_aln_fn)
  File "/data1/yuxiao/venv_for_mbin/lib/python2.7/site-packages/mbin/controls.py", line 40, in extract_controls
    opts           = controls.scan_WGA_aligns()
  File "/data1/yuxiao/venv_for_mbin/lib/python2.7/site-packages/mbin/controls.py", line 352, in scan_WGA_aligns
    reader = openIndexedAlignmentFile(self.control_aln_fn)
  File "/data1/yuxiao/venv_for_mbin/lib/python2.7/site-packages/pbcore/io/opener.py", line 52, in openIndexedAlignmentFile
    return CmpH5Reader(fname, sharedIndex=sharedIndex)
  File "/data1/yuxiao/venv_for_mbin/lib/python2.7/site-packages/pbcore/io/align/CmpH5IO.py", line 729, in __init__
    self._loadAlignmentInfo(sharedIndex)
  File "/data1/yuxiao/venv_for_mbin/lib/python2.7/site-packages/pbcore/io/align/CmpH5IO.py", line 745, in _loadAlignmentInfo
    rawAlignmentIndex = self.file["/AlnInfo/AlnIndex"].value
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/data1/yuxiao/venv_for_mbin/lib/python2.7/site-packages/h5py/_hl/dataset.py", line 313, in value
    "Use dataset[()] instead.", H5pyDeprecationWarning)
h5py.h5py_warnings.H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead.
(venv_for_mbin) [yuxiao@localhost data_tmp]$ python -V
Python 2.7.12

Still got the same error message. And I'm here hoping someone can help me. Thx a lot.

Answer 1

As you said, you need to use the new way of getting scalar values out. The .value syntax still works, but who wants to see pink warnings in Jupyter?

The message is not very intuitive, as isn't the syntax. So I will spell this out here - you have a d5py.File called f:

>>> f = h5py.File('temp.h5')
>>> f['path/to/scalar'] = 5
>>> f['path/to/scalar'].value
(...)\site-packages\h5py\_hl\dataset.py:313: H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead. "Use dataset[()] instead.", H5pyDeprecationWarning)
5

"Use dataset[()] instead" means literally "use the indexing (__getitem__) operator, and pass to it an empty tuple (()). So the last line should read:

>>> f['path/to/scalar'][()]
5

If you feel this is a bit of a strange, exotic notation, I can only empathize.

But it makes some sense - the numpy.shape of scalar values is in fact ().