thesweeheng's weblog

American Binomial Model in Python

thesweeheng — Fri, 11 Dec 2009 15:04:54 +0000

Having written about pricing American-style options on a binomial tree in q, I thought it would be instructive to do the same in Python and NumPy. Here is the code:

import functools as ft
import numpy as np

def BPTree(n, S, u, d):
  r = [np.array([S])]
  for i in range(n):
    r.append(np.concatenate((r[-1][:1]*u, r[-1]*d)))
  return r

def GBM(R, P, S, T, r, b, v, n):
  t = float(T)/n
  u = np.exp(v * np.sqrt(t))
  d = 1./u
  p = (np.exp(b * t) - d)/(u - d)
  ptree = BPTree(n, S, u, d)[::-1]
  R_ = ft.partial(R, np.exp(-r*t), p)
  return reduce(R_, map(P, ptree))[0]

def American(D, p, a, b): return np.maximum(b, D*(a[:-1]*p + a[1:]*(1-p)))
def VP(S, K): return np.maximum(K - S, 0)
ABM = ft.partial(GBM, American)

There is a minor deviation from the q code: we are allowing d to be specified in BPTree. But otherwise, they are doing the same thing. Performance (as measured in ipython) isn’t too far-off either:

In [1]: from binomial import *
In [2]: %timeit ABM(ft.partial(VP,K=102.0), 100.0, 1.0, 0.08, 0.08, 0.2, 1000)
10 loops, best of 3: 38.4 ms per loop
In [3]: ABM(ft.partial(VP,K=102.0), 100.0, 1.0, 0.08, 0.08, 0.2, 1000)
Out[3]: 6.2215001602514555

Note the similarity between the q and Python code. The similarity is a result of using NumPy and functools which enabled Python to perform array-oriented computation and partial function application. We did use a loop in BPTree as Python/NumPy does not seem to have the same “scan” operation as q. I suppose we could have created a numpy.ufunc to use accumulate()… but the loop felt cleaner and more Pythonic.

American Binomial Model in q

thesweeheng — Wed, 09 Dec 2009 15:22:04 +0000

American-style options may be exercised prior to expiry. To price them on a binomial tree, the full tree has to be constructed. The prices are then computed at each node through a backward reduction process starting from the prices at maturity. At each node, we take the maximum of the payoff at that node versus the price implied by the binomial model.

Constructing a binomial price tree is relatively easy in q:

BPTree:{[n;S;u] n{(x*y 0),y%x}[u]\1#S}  / binomial price tree

where n is the depth of the tree, S is the current price and u is the scale of the up-move. We simply let the scale of the down-move be 1/u and hence the y%x in the expression.

The general binomial model can then be implemented as follow:

GBM:{[R;P;S;T;r;b;v;n]                  / General Binomial Model (CRR)
 t:T%n;                                 / time interval
 u:exp v*sqrt t;                        / up; down is 1/u
 p:(exp[b*t]-1%u)%(u-1%u);              / probability of up
 ptree:reverse BPTree[n;S;u];           / reverse binomial price tree
 first R[exp[neg r*t];p] over P ptree }

where R is a reduction function, P is the payoff function, S is the current price, T is the time to maturity, r is the risk-free rate, b is the cost of carry, v is the volatility and n is the depth of the tree. For American and European options, the reduction functions may be expressed as:

American:{[D;p;a;b] max(b;D*(-1_a*p)+1_a*1-p)}
European:{[D;p;a;b] D*(-1_a*p)+1_a*1-p}

where D is the discount factor and p is the probability of an up-move. Consequently, we can express the American and European binomial models simply as:

ABM:GBM[American]
EBM:GBM[European]

Testing the code on an American vanilla put option (strike = 102; price = 100; time to maturity = 1 year; risk-free rate = cost of carry = 8%; volatility = 20%; depth of tree = 1000):

q) VP:{[S;K]max(K-S;0)}    / vanilla put: max(K-S,0)
q) \t show ABM[VP[;102];100;1;0.08;0.08;0.2;1000]
6.2215
31

It took 31 ms to compute. Pretty nice for so little code.

Note: It turns out that this implementation of EBM is faster than the one in the previous post. The reason is that I avoided using the expensive xexp function this time round. Otherwise, the previous implementation should be faster since it only computes the payoffs at maturity and not the intermediate nodes.

European Binomial Model in q and Python

thesweeheng — Thu, 03 Dec 2009 16:48:41 +0000

In the previous post, we created a binomial probability mass function (pmf). We can use that to easily evaluate European-style options:

EBM:{[P;S;K;T;r;b;v;n]                 / European Binomial Model (CRR)
 t:T%n;                                / time interval
 u:exp v*sqrt t;                       / up
 d:1%u;                                / down
 p:(exp[b*t]-d)%(u-d);                 / probability of up
 ns:til n+1;                           / 0, 1, 2, ..., n
 us:u xexp ns;                         / u**0, u**1, ...
 ds:d xexp ns;                         / d**0, d**1, ...
 Ss:S*ds*reverse us;                   / prices at tree leaves
 ps:pmf[n;p];                          / probabilities at tree leaves
 exp[neg r*T]*sum P[Ss;K]*ps }

Note that P is the payoff, S is the current price, K is the strike price, T is the time to maturity, r is the risk-free rate, v is the volatility, b is the cost of carry and n is the depth of the binomial tree. The Python version using NumPy and SciPy actually looks quite similar:

def EuropeanBinomialModel(P, S, K, T, r, b, v, n):
  n = int(n)
  t = float(T)/n                              # time interval
  u = np.exp(v * np.sqrt(t))                  # up
  d = 1/u                                     # down
  p = (np.exp(b*t)-d)/(u-d)                   # probability of up
  ns = np.arange(0, n+1, 1)                   # 0, 1, 2, ..., n
  us = u**ns                                  # u**0, u**1, ...
  ds = d**ns                                  # d**0, d**1, ...
  Ss = S*us*ds[::-1]                          # prices at leaves
  ps = binom_pmf(ns, n, p)                    # probabilities at leaves
  return np.exp(-r*T) * np.sum(P(Ss,K) * ps)

As we can see, both code has no explicit loops. This is possible in Python as NumPy and SciPy are array-oriented. NumPy and SciPy’s idea of “broadcasting” has some similarity with k/q’s concept of “atomic functions” (definition: a function f of any number of arguments is atomic if f is identical to f’).

Binomial distribution in q

thesweeheng — Thu, 03 Dec 2009 16:33:18 +0000

Recently I was using SciPy’s scipy.stats.binom.pmf(x,n,p). I though it would be great if I could have such a function in q. So a simple idea is to construct a binomial tree with probabilities attached. Recalling that a Pascal triangle is generated using n{0+':x,0}\1, I modified it to get:

q)pmf:{[n;p]n{(0,y*1-x)+x*y,0}[p]/1#1f}
q)pmf[6;0.3]
0.000729 0.010206 0.059535 0.18522 0.324135 0.302526 0.117649
q)sum pmf[1000;0.3]
1f

What is great about this method is that it is stable. Compared to SciPy 0.7.0, it was more accurate too (it is a known issue that older SciPy has buggy binom.pmf):

>>> scipy.stats.binom.pmf(range(0,41),40,0.3)[-5:]
>>> array([3.33066907e-15, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.11022302e-16])
q)-5#pmf[40;0.7]
3.293487e-15 1.52594e-16 5.162955e-18 1.134715e-19 1.215767e-21

Unfortunately this method is too slow for large n. For large n, we need more sophisticated methods. For the interested reader, take a look at Catherine Loader’s Fast and Accurate Computation of Binomial Probabilities paper and an implementation of a binomial distribution in Boost

GAE: Storing serializable objects in datastore

thesweeheng — Mon, 16 Nov 2009 14:00:52 +0000

Google AppEngine’s datastore supports a variety of simple Types and Property Classes by default. However if we want to store something like a dictionary, we typically have to serialize it and store it as a Blob. On fetching we will de-serialize it. While this approach works, it is repetitive and somewhat error-prone.

Wouldn’t it be great if there is a SerializableProperty class that can handle this automatically for us? It doesn’t exist but according to this article, it is easy to create our own customized Property classes. So here is a simple implementation of SerializableProperty that worked for me:

import cPickle as pickle
import zlib
from google.appengine.ext import db

class SerializableProperty(db.Property):
  """
  A SerializableProperty will be pickled and compressed before it is
  saved as a Blob in the datastore. On fetch, it would be decompressed
  and unpickled. It allows us to save serializable objects (e.g. dicts)
  in the datastore.

  The sequence of transformations applied can be customized by calling
  the set_transforms() method.
  """

  data_type = db.Blob
  _tfm = [lambda x: pickle.dumps(x,2), zlib.compress]
  _itfm = [zlib.decompress, pickle.loads]

  def set_transforms(self, tfm, itfm):
    self._tfm = tfm
    self._itfm = itfm

  def get_value_for_datastore(self, model_instance):
    value = super(SerializableProperty,
        self).get_value_for_datastore(model_instance)
    if value is not None:
      value = self.data_type(reduce(lambda x,f: f(x), self._tfm, value))
    return value

  def make_value_from_datastore(self, value):
    if value is not None:
      value = reduce(lambda x,f: f(x), self._itfm, value)
    return value

Usage is as simple as this:

class MyModel(db.Model):
  data = SerializableProperty()

entity = MyModel(data = {"key": "value"}, key_name="somekey")
entity.put()
entity = MyModel.get_by_key_name("somekey")
print entity.data

Hope that helps!

Update (20091126): I’ve changed db.Blob to self.data_type as suggested by Peritus in Comment. The same comment also suggested a JSONSerializableProperty subclass:

import simplejson as json
class JSONSerializableProperty(SerializableProperty):
  data_type = db.Text
  _tfm = [json.dumps]
  _itfm = [json.loads]

Thanks Peritus!

Gray code in q

thesweeheng — Sat, 12 Sep 2009 18:00:59 +0000

It is easy to construct binary n-bit Gray code in q using the recursive reflection-prefixing technique:

q)gc:{$[x;(0b,/:a),1b,/:reverse a:.z.s x-1;1#()]}
q)show gc 4
0000b 0001b 0011b 0010b 0110b 0111b 0101b 0100b
1100b 1101b 1111b 1110b 1010b 1011b 1001b 1000b

It is also possible to construct the above iteratively using the formula :

q).q.xor:{not x=y}
q)gc_iter:{(0b vs x) xor (0b vs x div 2)}
q)show (-4#gc_iter@) each til 16
0000b 0001b 0011b 0010b 0110b 0111b 0101b 0100b
1100b 1101b 1111b 1110b 1010b 1011b 1001b 1000b

To check that indeed exactly one bit is flipped each time:

q)check:{x[0] (sum@xor)': 1_x}
q)check gc 5
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

To identify the position of the bit that was flipped:

q)pos:{raze x[0] (where@xor)': 1_x}
q)pos gc 5
4 3 4 2 4 3 4 1 4 3 4 2 4 3 4 0 4 3 4 2 4 3 4 1 4 3 4 2 4 3 4

If we think about it, there is no reason why we have to prefix. We could do suffix as well:

q)gc:{$[x;(a,\:0b),(reverse a:.z.s x-1),\:1b;1#()]}
q)show gc 4
0000b 1000b 1100b 0100b 0110b 1110b 1010b 0010b
0011b 1011b 1111b 0111b 0101b 1101b 1001b 0001b
q)check gc 5
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
q)pos gc 5
0 1 0 2 0 1 0 3 0 1 0 2 0 1 0 4 0 1 0 2 0 1 0 3 0 1 0 2 0 1 0

In fact, if we are only interested in the positions that need to be flipped, we can use this instead:

q)gcpos:{$[x;a,n,a:.z.s n:x-1;()]}
q)gcpos 5
0 1 0 2 0 1 0 3 0 1 0 2 0 1 0 4 0 1 0 2 0 1 0 3 0 1 0 2 0 1 0

Such a sequence of positions is useful if we are using Gray code to efficiently enumerate the non-zero points spanned by a set of basis vectors:

q)basis:(1100000b;0111001b;0000011b)
q){x xor y} scan basis gcpos count basis
1100000b
1011001b
0111001b
0111010b
1011010b
1100011b
0000011b

Update (20090927): Once again, Attila has beaten me at q-golf :-) Here is his formulation:

gc:{x{(0b,/:x),1b,/:reverse x}/1#()}

Building SAGE 4.1.1 on Fedora 11

thesweeheng — Fri, 28 Aug 2009 13:14:46 +0000

While building SAGE 4.1.1 on a AMD Phenom II running Fedora 11, GCC 4.4.1 will hang the machine when compiling “base3.c” of PARI. Apparently it was sucking up all the available memory.

According to this thread, it happens on Ubuntu 9.10 too and the solution is to compile with -O1 instead of -O3 optimization. Unfortunately it wasn’t obvious (to me, at least) how to make GCC use -O1 specifically for PARI only.

Digging around in SAGE’s build system, I figured it could be done by repacking the PARI spkg with a modified “get_cc” script:

cd sage-4.1.1/spkg/standard
tar jxf pari-2.3.3.p1.spkg
sed 's/OPTFLAGS=-O3/OPTFLAGS=-O1/g' \
  pari-2.3.3.p1/src/config/get_cc > get_cc
mv get_cc pari-2.3.3.p1/src/config/get_cc
mv pari-2.3.3.p1.spkg pari-2.3.3.p1.spkg.orig
tar jcf pari-2.3.3.p1.spkg pari-2.3.3.p1

After that, I was able to compile SAGE using its standard build procedure. Admittedly this is a quick hack. A better solution may be to set OPTFLAGS according to the version of GCC used.

Updates: According to this thread, it is fixed in Ubuntu Karmic.

Grep binary string

thesweeheng — Thu, 20 Aug 2009 08:09:33 +0000

A co-worker asked me if it is possible to grep arbitrary binary strings, e.g. sequences of non-printable ASCII characters. It turns out that GNU grep does understand binary strings if we use Perl-regex via the -P option.

[sh@pc ~]$ grep -slrP '\x05\x00\xc0' /boot
/boot/grub/ffs_stage1_5
/boot/grub/ufs2_stage1_5
/boot/grub/stage2
/boot/efi/EFI/redhat/grub.efi
/boot/vmlinuz-2.6.29.6-213.fc11.x86_64

I couldn’t find this when Googling for “grep binary” so I thought I should pen it down here.

gdata.finance

thesweeheng — Sat, 06 Jun 2009 20:45:17 +0000

My contributions (gdata.finance module) to the Google Data APIs Python Client Library has been accepted into version 1.3.3 of the library.

A Conversation with Arthur Whitney

thesweeheng — Sun, 26 Apr 2009 21:25:19 +0000

Arthur Whitney was interviewed on ACM’s Queue magazine. Here’s a link to the article and a link to a local PDF copy.