Project Aardvark

Intern vs. Compiler

Tuesday, May 31, 2005 posted by Benjamin Pollack

OK, in Tyler’s last post, he hinted that there were some segfaults involved getting a certain GPL program to run properly in VS.NET 2003. I want to elaborate on what exactly happened, because it's happened enough hours ago now to be funny instead of humiliating. Note that this is going to get a bit technical and I probably will come off looking like an idiot, in which case all I can say is a) sorry and b) I’m still learning Windows and VisualStudio, so cut me some slack. :)

On Friday afternoon, having pushed as far ahead with the reflector as I could until we had some of the other stuff that needs it actually running, I downloaded the GPL program we’ll be extending, discovered that it had Visual C++ project files, and opened those in VS.NET. I checked out the README, which said things ought to build just fine on VisualStudio.NET 2003, so I thought it would be a piece of cake to get running. Bzzzt! I hit F5, the build starts, and then dies with my least favorite error message in the entire world:

“Internal Compiler Error”

Just in case you have never had the pleasure of getting this error message (and given that we’re currently getting a lot of Joel on Software readers here, that’s probably unlikely), it means that the compiler not only cannot handle whatever code you gave it, but cannot even give you a meaningful error message, because the compiler itself has a bug. Worse, I didn’t get this error in just one lone file; I got it in almost a hundred different files in the project. So, I began spelunking around the source code. After about 45 minutes of tweaking code, sometimes getting further, other times segfaulting the compiler rather spectacularly, I finally realized that all of the errors were coming from C++ header files. On a hunch, I went to the build settings disabled precompiled headers. I started the build cycle again, and, about 25 seconds later (I love these computers) had...missing symbol definitions. Well, fine, but at least missing symbol definitions I can handle. A quick check revealed that the missing symbol definitions happened because the projects were simply missing a dozen or so .cpp files. I quickly threw those into the project and rebuilt, and then got something that genuinely did not make any sense to me.

“Symbol deflateEnd@4 not found.”

The symbol I recognized; the project has a dependency on zlib, and deflateEnd looked like (and is) part of zlib’s API. Yet I could trivially verify that zlib was building properly and getting linked properly. The problem, I decided, was that deflateEnd@4 was clearly a mangled name (i.e., the name that was compiled into the library did not match its actual name), and the only time I’d seen names mangled like that was from C++ projects, so VS.NET must be compiling zlib as C, but the main program thinks it’s C++. (Experienced Windows programmers are now laughing maniacally; the rest of you can just follow along.) So I spent another 30 minutes trying to figure out why zlib’s extern “C” command in the header file wasn’t taking hold until I finally gave up and asked for Joel’s help. Joel comes over, identifies that this is not C++ name mangling, but rather Windows __fastcall mangling, and quickly generates a nice printout of the symbol table of zlib.lib where we verify that there are (surprise) only normal C names. Thankfully, I now had enough information to fix the problem. I changed zlib to be compiled with __fastcall, then (when that failed) as __stdcall, and finally, finally had something that compiled.

Well, almost. Turned out I needed to link against some parts of Windows that the project didn’t specify. But after that it compiled. Cleanly.

It just segfaulted immediately on launch.

At that point, I was too worn out to continue and decided to just leave the whole mess for later. I checked it into our Subversion repository with a note that it crashed instantly on startup and went home for the weekend.

So today, while we were trying to get VS.NET and friends installed on Michael and Tyler’s computers, Michael very kindly came over and started attacking the program with fresh eyes. He succeeded in recreating the full call stack at the time of the crash (which was actually surprisingly difficult due to threading issues), and with that boost I was able to determine that the crash was happening due to an instance variable getting trampled, which was happening due to a call to a string copy that seem to be overwriting most of a class’ data.

So Michael and I are sitting there, hunched over the monitor, trying and failing with idea after idea, when Tyler wanders over and asks, “Is this the same as what’s in Subversion?” I tell him yes. He goes over, opens the project, builds it, and hits the Debug button.

And it runs.


We still have absolutely no idea what the problem was. I ended up blasting my entire source tree and checking out again from our repository, which ought to have been exactly the same broken code that I was working with before (I’m the one who checked it in, after all), but which now inexplicably works just fine. So now I’m finally able to hack on actually extending and modifying this stuff instead of waging a war with the compiler. I just feel annoyed, because I wasted several hours and I caused Michael to waste several hours when we could simply have switched to another machine. It wasn’t a complete waste of time—my initial project fixes were necessary (I double-checked), and I learned an awful lot about the VS.NET debugger—but I still feel like I could have gotten a lot more done if I’d just tried things on a different computer.

There’s probably a lesson to be learned here, but I’m not sure what it is, so instead I’ll just close by promising to keep you up-to-date if I discover my machine is possessed by demons. If it’s not, of course, you’ll have to make due with normal Project Aardvark updates. But here’s to hoping.

First Day Fun

Tuesday, May 31, 2005 posted by Tyler Griffin Hicks-Wright

The first day is going well so far.  First, we settled on desks and got all our software installed.  Next was the official Project Aardvark Kick-Off meeting, where Joel drew on the whiteboard for a while, we all nodded in agreement, and Lerone circled us with the camera.  After some pizza, we resumed getting our development environments set up, and Michael and Ben started hacking away on some open source software.  Ben has done a good deal of work getting the project files set up for .NET, but he was running into a segfault when the program started up.  After a little while of them beating on it, I decided to take a look on my machine.  I checked out the code, built it to check for compile errors, and then started the debugger.  Surprisingly, it worked -- no segfault occurred.  It turns out something in the project files got messed up on Ben's machine, so a clean checkout later he was up and running again.  It's just another testament to how finicky these things can be sometimes.  That's why we have source control.  :-)


Tuesday, May 31, 2005 posted by Yaron Guez

Well, I'm all moved in at the Marymount Manhattan dorm and am currently sitting at my new desk at Fog Creek.  It's looking to be a pretty drastic adjustment getting used to living in the city, but overall I'm pretty excited about all of it.  We just finished the first meeting, going over the specs and targets for Project Aardvark.  This is going to be a great product.  Joel gave us all stacks of books to read.  I'm the marketing guy so my stack consist of books on management, positioning, psychology, and (gasp) marketing.  We also got a calendar of events planned for us.  It looks incredible.  A welcome party, walking tour, museum tour, Yankees game (Sox fan...Yankee hater :-P), film festival, cruise, and several shows (including Spamalot!)....this is going to be quite the summer.

So...yeah, it'll be interesting to see what my day-to-day workings consist of.  For starters I'm going to get through these books and do some research on the competition.

Reflector: Feature-Complete

Thursday, May 26, 2005 posted by Benjamin Pollack

For the past two and a half days, I’ve been methodically chopping away at one of the four major components in Aardvark. The experience was a bit exhausting; I went from knowing no C# and only a limited amount of .NET at midmorning on Tuesday to basically hitting feature-complete on the component called the reflector at 1:00 PM Eastern. Now, I’ll be the first to admit “feature complete” is a code word for “still has a ton of small yet annoying bugs that you ‘forgot’ about for the time being, and at least one major bug that, if properly triggered, will cause your computer to explode” (see antonym: “finished”), but the fact that I was able even to get that far that fast attests to two things: the power of a good development environment and the value of a good product specification.


As you have all probably inferred from Joel’s photo of all the books we have to read, Aardvark will largely be implemented in C#. I initially approached C# with a bit of trepidation. Although my bread and butter (and grades) depend on C++ and Java, anyone who knows me well knows that I’d much rather be working in one of the dynamic languages—especially Squeak Smalltalk or Ruby—than in any of the C descendents. My previous experiences with Microsoft languages (VisualBasic in all of its many varieties) did not exactly raise my expectations, and I was prepared for the worst. After a few days, though, I can honestly say that C# basically feels like Java with many of the problems removed and surprisingly few new issues of its own. The .NET library, too, has been very solid, and several components whose counterparts only just arrived in Java 5 (such as relatively easy-to-use non-blocking I/O) greatly expedited my work. I’ve also begun playing with Mono—an open-source/free-software implementation of .NET for Windows, Linux, and Mac OS X—on my PowerBook, and have been fairly impressed. .NET, at least in its non-graphical variety, seems likely to become genuinely multi-platform. The fact that .NET is so similar to Java in so many ways does mean that it is not revolutionary by any stretch of the imagination, but even an incremental improvement can greatly aid productivity.


Yet VS.NET was not the tool that most aided my coding. That honor goes to the Aardvark product specification. Last semester, I served as an undergraduate teaching assistant in a course that is supposed to teach advanced program design. As part of the course projects, students had to write detailed specifications describing how their project would work. A lot of the students loathed writing these specifications, and generally wrote as little as possible. They then plowed in with half-finished designs, changed everything as they discovered they had failed to account for basic functionality, and ended up with a product that had little relation to their original proposal. The end result usually worked; it was even frequently well-designed. But the students went through a lot of needless pain by not planning ahead, and then having to go back and rewrite large amounts of code.


The Aardvark spec, in contrast, is exceedingly detailed, and makes crystal-clear how each discrete component should interact with the others. Quick sketches of screenshots show how a user will interact. Protocol specifications detail precisely how various components will communicate. Questions that cannot be easily answered without running prototypes are prominently flagged so that we know where we will experience potential problems and can prepare for them with flexible code. That does not mean that all of the decisions are already made for us at all; the reflector clocks in at 950 lines of code, and designed its internals on my own. We’ll all have to do likewise on the other components as well. What it does mean is that we can write the components in isolation. Once I finish writing a solid testing framework (probably using NUnit), use that to iron out the remaining bugs, and do one last empirical verification that everything is working properly according to spec, I can know that the reflector will work when it’s dropped into the rest of Aardvark—even though none of those components currently exist.


That’s power.


Next week: details on New York City (short version: it rocks) and a discussion of bringing the REPL to .NET.

Getting Ready to Start

Tuesday, May 24, 2005 posted by Benjamin Pollack

Well, Project Aardvark may not officially start until May 31st, but I’ve already arrived in New York City and am working at Fog Creek to ensure that we can actually start on the 31st. I’m the first one here, which means that I have a lot of unique opportunities: I get first pick for where I’ll sit, which potted flower will grace my desk, and which chair I get. The flip side of that is that I also had the opportunity to unpack the 30+ boxes of computer equipment you saw in the photo in Joel’s last post Yesterday, I set up five computers, ten LCDs, a router, lamps, flowers, tables, chairs, networking cables, extension cables, and more, all under blazingly hot lights setup by Lerone so he could tape the whole thing. Oy.

The good news is that that’s all done now (this morning I even finished getting the last of the boxes collapsed and shoved into the garbage area), and the reward is that I’ve got an insanely fast computer and a pretty decent view of western Manhattan in an awesome office. I even got enough of my computer’s development tools set up to accidentally publish an article to our live server when I was testing out CityDesk. It’s truly a great work environment.

I’ll be spending the rest of the day (and probably the rest of the week) doing some extremely preliminary research and development of Aardvark and actually getting back up-to-speed on VisualStudio.NET and C#, neither of which I’ve worked with very much in the past, as well as reading in and out of the pile of books sitting on my desk. Once I’ve gotten a little bit more settled in, I’ll post on how .NET development compares to Eclipse in Java and Xcode in C/C++/Objective-C.

Meanwhile, it’s time to get back to work!

The Goods

Thursday, May 19, 2005 posted by Joel Spolsky

Getting ready for Project Aardvark!

Piled up in our office, you see Aeron chairs, desks, Dell Workstations (Dual-Xeon, 2GB RAM, 150GB of RAID storage, Gigabit Ethernet), flat screen monitors (dual 20" for each computer), and UPSes.

Our summer interns will have some light reading to do before they can get started:

That's the Aardvark spec on the top, and a few books that I thought might be helpful. Standard issue for all new employees here includes a copy of Code Complete and a digital stopwatch (gotta make things fast).



Donald? Nah!

Wednesday, May 11, 2005 posted by Joel Spolsky

One day after the site goes live we're already in the news!

CNET "Could Spolsky be another 'Donald' in the making? Not likely."

Project Aardvark is officially beginning May 30th. You can subscribe to our RSS feed to keep up-to-date on our progress!

First Post!

Tuesday, May 10, 2005 posted by Joel Spolsky

Yes, it's true... there will be a filmmaker, Lerone Wilson of Boondoggle Films, working out of the Fog Creek office this summer creating a documentary about our summer internships.

As I wrote earlier, "This summer, Fog Creek Software has hired four summer interns from Yale, Duke, and Rose-Hulman. Our selection process was extremely competitive, with over 800 kids applying for only four positions.

"Instead of wasting their talents giving them the usual dull and unimportant tasks of a typical summer internship, we decided to let the interns create a complete new software product, from beginning to end, over the course of one summer. With experienced software developers as mentors, the team will design, program, test, and roll out a complete software product over the course of one hectic summer, going from concept to paying customers in about ten weeks."

Now, I'll be the first to admit that this product will be a 1.0 version... the minimum thing that can possibly work. We've thought long and hard about the amount of code it will take, and we're fairly confident that three interns can get a beta version out in about 4 weeks. One chunk of the project is basically an enhancement to the "fogshop" ecommerce engine; another chunk is a couple of simple features added onto an existing open source program; the third chunk is possibly the simplest sockets-based server you can imagine that still does something useful. Each chunk would be a reasonable programming assignment in Yale CS323 or MIT EECS 6.001 (insiders will recognize these as two of the most challenging programming courses on the planet, but, hey, we hire the kind of people who love challenges).

The fourth intern is going to focus on product marketing so he will be working on a brand name, pricing, the website, advertising, and PR. The product we're building is something we need internally anyway, so it's almost worth the expense even if nobody buys it, but we think it should be useful to a lot of other people, too, so I see this as fairly inexpensive market research.

We'll keep a weblog so you can track the interns' progress, and, if we're lucky, at the end of the summer there will be a DVD documentary you can get to see the whole thing in living color.

Special thanks to Mark Lemmons and his crew at Thought Equity, who have offered to loan us an HDV camera for the production of this movie!

About Project Aardvark

Fog Creek Copilot
Aardvark'd Movie

Instead of wasting their talents giving the the usual dull and unimportant tasks of a typical summer internship, we decided to let the interns create a complete new software product, from beginning to end, over the course of one summer. Read More