Tutorial

In [1]: import pandas as pd

In [2]: from pandas_msgpack import to_msgpack, read_msgpack

In [3]: df = pd.DataFrame(np.random.rand(5,2), columns=list('AB'))

In [4]: to_msgpack('foo.msg', df)

In [5]: read_msgpack('foo.msg')
Out[5]: 
          A         B
0  0.841185  0.075480
1  0.135736  0.496043
2  0.688984  0.500175
3  0.498896  0.727207
4  0.627269  0.398565

In [6]: s = pd.Series(np.random.rand(5),index=pd.date_range('20130101',periods=5))

You can pass a list of objects and you will receive them back on deserialization.

In [7]: to_msgpack('foo.msg', df, 'foo', np.array([1,2,3]), s)

In [8]: read_msgpack('foo.msg')
Out[8]: 
[          A         B
 0  0.841185  0.075480
 1  0.135736  0.496043
 2  0.688984  0.500175
 3  0.498896  0.727207
 4  0.627269  0.398565, 'foo', array([1, 2, 3]), 2013-01-01    0.095020
 2013-01-02    0.887087
 2013-01-03    0.704464
 2013-01-04    0.809950
 2013-01-05    0.509674
 Freq: D, dtype: float64]

You can pass iterator=True to iterate over the unpacked results

In [9]: for o in read_msgpack('foo.msg',iterator=True):
   ...:     print(o)
   ...: 
          A         B
0  0.841185  0.075480
1  0.135736  0.496043
2  0.688984  0.500175
3  0.498896  0.727207
4  0.627269  0.398565
foo
[1 2 3]
2013-01-01    0.095020
2013-01-02    0.887087
2013-01-03    0.704464
2013-01-04    0.809950
2013-01-05    0.509674
Freq: D, dtype: float64

You can pass append=True to the writer to append to an existing pack

In [10]: to_msgpack('foo.msg', df, append=True)

In [11]: read_msgpack('foo.msg')
Out[11]: 
[          A         B
 0  0.841185  0.075480
 1  0.135736  0.496043
 2  0.688984  0.500175
 3  0.498896  0.727207
 4  0.627269  0.398565, 'foo', array([1, 2, 3]), 2013-01-01    0.095020
 2013-01-02    0.887087
 2013-01-03    0.704464
 2013-01-04    0.809950
 2013-01-05    0.509674
 Freq: D, dtype: float64,           A         B
 0  0.841185  0.075480
 1  0.135736  0.496043
 2  0.688984  0.500175
 3  0.498896  0.727207
 4  0.627269  0.398565]

Furthermore you can pass in arbitrary python objects.

In [12]: to_msgpack('foo2.msg', { 'dict' : [ { 'df' : df }, { 'string' : 'foo' }, { 'scalar' : 1. }, { 's' : s } ] })

In [13]: read_msgpack('foo2.msg')
Out[13]: 
{'dict': ({'df':           A         B
   0  0.841185  0.075480
   1  0.135736  0.496043
   2  0.688984  0.500175
   3  0.498896  0.727207
   4  0.627269  0.398565},
  {'string': 'foo'},
  {'scalar': 1.0},
  {'s': 2013-01-01    0.095020
   2013-01-02    0.887087
   2013-01-03    0.704464
   2013-01-04    0.809950
   2013-01-05    0.509674
   Freq: D, dtype: float64})}