This is where I bitch about serialization in C++. If you do not understand what that previous statement was about, you might want to skip all of this entry. Really, skip this entry, it does not help you with much. Yes, I am home for the winter, I am leaving in a couple of days. It was good to be home, this is my last winter break for a while. But really, this entry is about serialization, maybe somewhat about reflection, but mostly about serialization. Leave it alone, it doesn't link to that much even.

Before I begin bitching, there are a couple of things that need to be put out. Yes I am aware of the existence such things as s11n, and the Boost Serialization library. I have even spent a few days looking at such things as Thrift, and the promise of writing something like an IDL, passing it through a compiler, producing C++ code which then serializes itself. Really this is a great idea, and if I was worried about writing serialization code for myself I'd have picked one of those. Overall, I would probably have gone with s11n for things I was adapting to be serializable, and Thrift for everything else. But then again, I could also use Python, Java, C# or any of the dozen languages which provide nice serialization interfaces.

The thing is I am not writing code for myself, well I am, but not really. I am trying to see if this class I am TAing can be made doable in C++. I don't like all of those solutions for various reasons, listed forthwith.

1. s11n, boost: I think any serialization depending on me completely specifying what needs to be serialized in code has issues. In short, there are times when data fields are added to classes, and people forget to change code. This is somewhat panful to debug (think of forgetting to initialize stuff in a constructor and such), adding another place where you need to worry about this seems like a poor decission.
2. Thrift, ...: Learning another language, merely for the privilege of serialization seems a little counterproductive. Besides why pick Thrift's IDL over any of the other serializable language. After all managed code is supposed to become as fast as native code, give or take a little

No sir, I do not like either of those choices, and for once in my life, I find myself wanting something lots of languages have, in C++, because pig headedly this is the Java alternate some of us backed.

s11n's author aptly states that serialization is easy, it is deserialization that really sucks. Assuming we are not going down blind alleys (as I once was), packing data in a known format, with know boundaries, is a fairly reasonable strategy of assuring oneself that the data seen on the receiving end did indeed originate from a compatible library. You ask what of data which has boundaries and properties identical to what you're expecting, but is actually corrupt. Well no matter how much you worry about solving problems involving arbitrary data, one must always remember that provably showing equivalence between two generators of data is unsolvable.

Now the only real reason deserialization is so hard, is because in some cases you want people to be able to encapsulate arbitrary data in objects belonging to your class. Arbitrary data is somewhat hard to dynamically recreate. And by somewhat I mean a lot.

Let us assume then that we neither want to learn an IDL, nor do we want to face the problems I feel exist with using libraries like s11n. However, we also recognize the fact that s11n or boost is probably much better done than anything we can produce, and perfect in a reasonable amount of time (a few hours in particular). Now if we had reflection, or what some fancy sites seem to call meta-object programming, we could go through the process of producing code which automatically serializes all data fields in an object. Well C++ doesn't really have reflection, and C++'s version of RTTI is not very useful for these purposes.

But we are stubborn, pig headedness always comes with stubbornness. Well if we had multi-pass compilers, we could use C++'s macro system (which is arguably not very powerful), to build ourselves a reflection framework, and then use said reflection framework in the way mentioned above. After all, we know that C/C++ macros are mere text replacement tools, and hence it makes perfect sense to write #defines which produce other #defines. Of course the C++ compiler is single pass, probably for many many reasons, including the avoidance of infinite loops. Ah the avoidance of infinite loops, always a worthy cause.

Well I am lost. I have ideas, none of them seem to work. Writing serialization code by hand seems to be very very painful, and is one of those things I know is going to come back to haunt me later this semester. We need better alternates to Java. And other things...

Panda

I believed I had this beautiful set of numbers. Then reality struck and they weren't actually there. On one hand, having fake but nice numbers is bad, on the other hand, those numbers were really pretty. And now I sit around reading and thinking about Haskell and hating on C++. Who would have known that j would differ in

 
  std::vector<int> something[numelts];
  something[7].push_back(2);
  int j = something[7][0];
 

and

 
  std::vector<int> something[numelts];
  something[7].push_back(2);
  int j = (something[7])[0];
 

.
The later is correct, the first is wrong, C++ seems to be treating unparenthisized things as normal array lookups, while the parenthisized thing leads to it using operator[] from vector.

panda

So I finally have a fairly stable install of FreeBSD on my Mac Mini, the only noticable problem I have had so far is a weird problem with running X forwarding on it, since it somehow causes reboots, though no real kernel panics, since db doesn’t come up, and I have no real core dumps to look at. So it pretty much works out normally, install minimal with docs, manpages, and other stuff, use ports and such, to install X, and it mostly works. The only thing I really had to change was drivers for the network card, the Mac Mini uses a Marvell YukonII NIC, and well Marvell/SysKonnect released the driver code under a FreeBSD license, and it is available here. Unfortunately this does not work as such, and you have to patch if_yk.c because one of the system calls changed. Applying the patch


1569,1570c1569,1570
< }
< #else
---
> }
> #elif __FreeBSD_version < 700006
1577a1578,1588
> SK_ADDR_LOGICAL_ADDRESS);
> }
> #else
>
> if (bcmp(IF_LLADDR(pAC->pIfnet),
> pAC->Addr.Net<sup><a href="#fn0">0</a></sup>.CurrentMacAddress.a,6)) {
>
> SkAddrOverride(pAC,
> pAC,
> 0,
> (SK_MAC_ADDR*)IF_LLADDR(pAC->pIfnet),


should do the trick.

Ze Panda

So people on like half a dozen Python forums quibble about how the socket.ssl library for python doesn’t allow for certificate checking, and such, and well since I have really been spending miniscule amounts of time on this project anyways (like 30 mins to an hour a day), I figured I’d get a better OpenSSL implementation in. Be aware, that because I am using pyOpenSSL, I need to make this LGPL, rather than the usual Python license. To use this version of the module, you willl need pyOpenSSL, and perhaps OpenSSL, I refuse to distribute cryptographic libraries directly on this website because of all the legalese involved, all the export control requirements, and such. It is so much easier to let other people take care of that.

Ze Panda

PS: Forgot to link to the module. A bundle with the module and sample code is available here

I have a bunch to write about a recent surge in python use for me, some of the cool things I played with, and using Python for testing weird C programs. But it is late, and I need to get to sleep soon, so I am just going to put something less harmful, and less likely to require me to figure out legal tangles.

I have been working on something involving NNTP, mostly a minor project to create a few very simple command line NNTP tools, in part so I can pipe these together and create some sort of a NNTP-IMAP bridge, and perhaps a NNTP-RSS bridge, I preffer the IMAP idea, but well this is a little more in the future. Anyways, I spent some time creating a monitor, that I am not going to post quite yet, since I want to improve it a little, and one of my pet peeves with the python lib was that it provided fairly good NNTP support, but no NNTPs support, and it is sort of hard to extend the NNTP class itself to provide NNTPs support, because <i>init</i> seems to call on a few receive methods before returning, and this leads to some problems with where to put the ssl initialization code. Now seeing as Brown’s news servers, something I use pretty frequently because of internal newsgroups, runs on SSL, this was something which was bugging me. So I sort of took the library as it existed, and modified it to do NNTPs, and these are fairly minor changes, and my version of the library is completely backward compatible with nntplibrary’s current non-ssl functionality. There’s a patch submitted on sourceforge, but seeing as that’s going to take some time I am putting a tar with both the library (nntpslibrary, so I don’t have to worry about overriding someone’s nntplibrary) and a test program which doesn’t do much, but shows the basic steps for connecting.

This code is not covered by the Creative Commons license for the blog, and is distributed as is, without waranty, under the Python 2.4 license.

File