Tutorial

In [1]: import pandas as pd

In [2]: from pandas_msgpack import to_msgpack, read_msgpack

In [3]: df = pd.DataFrame(np.random.rand(5,2), columns=list('AB'))

In [4]: to_msgpack('foo.msg', df)

In [5]: read_msgpack('foo.msg')
Out[5]: 
          A         B
0  0.194458  0.600776
1  0.420979  0.950423
2  0.522773  0.380947
3  0.319841  0.734287
4  0.111098  0.998534

In [6]: s = pd.Series(np.random.rand(5),index=pd.date_range('20130101',periods=5))

You can pass a list of objects and you will receive them back on deserialization.

In [7]: to_msgpack('foo.msg', df, 'foo', np.array([1,2,3]), s)

In [8]: read_msgpack('foo.msg')
Out[8]: 
[          A         B
 0  0.194458  0.600776
 1  0.420979  0.950423
 2  0.522773  0.380947
 3  0.319841  0.734287
 4  0.111098  0.998534, 'foo', array([1, 2, 3]), 2013-01-01    0.062266
 2013-01-02    0.448738
 2013-01-03    0.421353
 2013-01-04    0.727311
 2013-01-05    0.089788
 Freq: D, dtype: float64]

You can pass iterator=True to iterate over the unpacked results

In [9]: for o in read_msgpack('foo.msg',iterator=True):
   ...:     print(o)
   ...: 
          A         B
0  0.194458  0.600776
1  0.420979  0.950423
2  0.522773  0.380947
3  0.319841  0.734287
4  0.111098  0.998534
foo
[1 2 3]
2013-01-01    0.062266
2013-01-02    0.448738
2013-01-03    0.421353
2013-01-04    0.727311
2013-01-05    0.089788
Freq: D, dtype: float64

You can pass append=True to the writer to append to an existing pack

In [10]: to_msgpack('foo.msg', df, append=True)

In [11]: read_msgpack('foo.msg')
Out[11]: 
[          A         B
 0  0.194458  0.600776
 1  0.420979  0.950423
 2  0.522773  0.380947
 3  0.319841  0.734287
 4  0.111098  0.998534, 'foo', array([1, 2, 3]), 2013-01-01    0.062266
 2013-01-02    0.448738
 2013-01-03    0.421353
 2013-01-04    0.727311
 2013-01-05    0.089788
 Freq: D, dtype: float64,           A         B
 0  0.194458  0.600776
 1  0.420979  0.950423
 2  0.522773  0.380947
 3  0.319841  0.734287
 4  0.111098  0.998534]

Furthermore you can pass in arbitrary python objects.

In [12]: to_msgpack('foo2.msg', { 'dict' : [ { 'df' : df }, { 'string' : 'foo' }, { 'scalar' : 1. }, { 's' : s } ] })

In [13]: read_msgpack('foo2.msg')
Out[13]: 
{'dict': ({'df':           A         B
   0  0.194458  0.600776
   1  0.420979  0.950423
   2  0.522773  0.380947
   3  0.319841  0.734287
   4  0.111098  0.998534},
  {'string': 'foo'},
  {'scalar': 1.0},
  {'s': 2013-01-01    0.062266
   2013-01-02    0.448738
   2013-01-03    0.421353
   2013-01-04    0.727311
   2013-01-05    0.089788
   Freq: D, dtype: float64})}