ThePanda

So anyone know why allocation of largish lists seems to take so long in Python, like time measurable in seconds for largish sizes (about ten million or so). Like if they were doing it the obvious way (taking into account the fact that someone might actually be doing something like [0 for x in xrange (0, maxwt)]), they would for one realize that someone’s just about to allocate maxwt-1 elements (how about being able to specify this, I don’t mind hinting that this is going to happen, or even giving it to them), and be using something which allocates increasingly larger amounts of memory as you run out of space. It is an amortized O(1) operation if you keep doubling your memory heap, and seeing as the python allocator runs out of memory soon enough anyways, there seems little danger that this would lead to too many bad things. Or their could be an easier way to pass this information to the execution environment. Either ways, lack of fast array allocation has me disappointed.

And before supporters of OCaml (the other language I like), comment on this, OCaml arrays are in my opinion too constrained by the maximum allowed size, and BigArrays need a better interface.

Of course I could just use C for code which builds partial tables for large problems using DP, but umm I wanted to use the freedom offered by a grad seminar.

Ze Panda

This entry was posted on Thursday, February 15th, 2007 at 2:34 am and is filed under article. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.