Got an iPhone

Yep, just like 1 million other people I got an iphone the monday after they came out.

I was 100% sure about getting one by mid July.

There is already a lot of commentary, I’ll give you my 2 week take on the device. First, a little background. I worked at Danger Research back in 2000/2001. I sat in on the meetings. I learned a lot about what it takes to get a device out into the market. Some of the Danger team has been at Google and they, I suspect have been working on the fabled Google phone.

Apple won. They have blown everybody away.

This was a big jump for Apple. The cellphone part of this has lots of pitfalls. They nailed it.

  • It works really well.
  • No other phone has an interface this good.
  • No other phone has a web browser this good.

Overall, this is hands down the best phone I’ve ever had. I use the phone, contacts, ipod, and the browser heavily.

I have a few nits with the Windows side of the house, but I’ll cover that in another post.

Before I finish, here are some key differentiators that I don’t hear about at other sites:

They can upgrade your software with iTunes Sure, you could get a firmware upgrade for a phone at a cellular store, but that isn’t the same. They never expected to do those en-masse. Apple knows that everytime someone syncs, they have opportunity to upgrade. Whatever complaints people have could get addressed in the next update.

Apple gets the crash reports No other phone sent their crashes to the manufacturer. Apple will fix any bugs as they come up. So far any ‘crashes’ that I’ve witnessed were just apps disappearing and reappearing. This wasn’t a blue screen or a deadlock.

There isn’t an ATT portal for the device. This is an internet device. The mobile provider lost this power struggle. They are Apple’s customer. Contrast that with prior devices and the mobile networks.

Everybody thinks 3G is easy and that the carriers have their act together. They don’t. They have tiny data connections to their towers. They are just recently pulling off voice in a consistent manor (no more drops).

Posted in General | Comments Off on Got an iPhone

Blade Runner Final Cut in Theaters Fall 2007

Blade Runner

Wow – after my prior post about the 25th Anniversary, I read THIS at the Big Picture!

Wow! Might have to make it down to San Diego for the ComicCon on the 27th.

Also, the other night I spent about half-hour just reading this site on cyberpunk. One of the posts by the site-owner in the archive mentioned something interesting. He/She said all of the best stuff came out in the 80’s. I wonder why that happened.

Posted in General | Comments Off on Blade Runner Final Cut in Theaters Fall 2007

I bought a Prius

Prius

I am commuting a lot to the city, but I was driving a not so efficient Pilot. The wife was driving the Odyssey. After a service on the Odyssey around January of this year, we discovered that it needed a lot of work. At the same time Toyota had some great incentives for the Prius. My wife also needed the extra seating that the Pilot had for some kid carpooling. It was the right opportunity to get a Prius.

In order to find the best price, I did a lot of online comparision shopping. I ended up using cars.com and having dealers send me quotes. If you know what you want, they will be pretty responsive. It turns out that a lot of dealers had a lot of inventory, which was in contrast to the prior sales history for the Prius. After haggling with 2 different fleet managers, I went in and bought the car at Magnusson’s in Palo Alto.

Interesting coincidence, the sales guy kept a book of the past buyers that were famous. I found out that this guy sold a Prius to Andy Hertzfeld, and had his signature. This didn’t break my concentration, we still had to make it through the finance department upsells. It was just interesting.

Anyways, we got one and I’ve been driving it for about 5 months. It’s a great car and the mileage is excellent! The tech in the car is better than anything else I’ve ever seen. Also, the NAV is so good. I will always have that in any future car. Overall, fantastic car.

Posted in General | Comments Off on I bought a Prius

Windows without VMware for development no longer makes sense

When I started doing Ruby on Rails work, I was using a Mac at home. When I left Plaxo, and started consulting, I needed a cheap laptop. I got an inexpensive Dell / Windows laptop. I have a lot of experience with Windows, and I have some really good developer tools on the Windows side. When I wanted to do some Rails on the laptop, I found a windows package that gave me a full Rails setup. For a while this was great.

Windows is caustic to open source libs

When it was just me, I deployed from the Windows laptop to the Unix cluster. Once I started integrating with the developers that we hired at work, things started to break down. They were all on Linux, and my checkins started screwing with their editors (linefeed issues). The really big problem was that certain gems and packages didn’t exist on windows. I’ve developed libraries in the past and I can understand that many library authors don’t want to deal with windows. Microsoft doesn’t have a decent, free environment for building open source software. Even with Cygwin, all sorts of things break. It is a hard platform to support so most of them don’t do it well. What could I do?

The Server

Oh yeah, just to be explicit, there is no way we’re going to run a web app on anything other than Linux. It is a commodity. Another Unix would be possible and fine, but we don’t have that need at this point. Would we ship on windows? Not even if you paid us. Windows has been irrelevant on the server side for so long. It is too opaque, too proprietary, has licensing issues (how many sockets am I allowed to open?), build issues (see above paragraph). The Unix API has won, its free, and that is where all the action is.

Options

Windows is a fine desktop, but I needed a solution. I had free Unixs as my desktop in the past, and they weren’t and still aren’t ready yet. I needed to be productive, so I wasn’t going to go back to that. Also, we are a startup, I’m not about to drop $4k on a Mac laptop right now. Finally, I’m not going to deal with driver hell with Linux on a laptop. As I was researching a solution, an old friend from Plaxo, Ethan, mentioned that VMware had a free VM player. After thinking about it, this looked like a really appealing solution. I only needed a Linux box that I could telnet/ssh into. I could have a real windows environment, yet have a little Unix server running inside. This looked like the ideal, no compromises for either setup.

VMware Player Rocks

I gave it a try, and in under an hour it was working really really well. The CPU hit wasn’t noticeable at all. I’m so pleased with the solution. To any of you (the two people out there reading this), if this was obvious to you a few years ago… I apologize for my slow uptake on this. I’m a believer now. Seriously, even if I was on a Mac, I could imagine running a little Linux distro in Parallels just to be compatible with my production environment. In fact, if I moved to a Mac, I could probably keep the same VMware image!

Windows Libs Are No Longer Relevant

About 9 months later, this setup has been great. Today though, it really hit me. I still see Ruby gems and modules for other languages supporting windows. Why? I can say that if I were developing an open source library, that was useful in a server/web environment, I would drop support for the hardest environment to support: Windows. People shouldn’t use Windows for a server. With the free VM systems out there, they don’t even need it for their windows dev box. If the library developers out there didn’t have to support Win32 anymore, how much lower of a load would that be? Something to think about.

If anybody complains, point them at this post ๐Ÿ™‚

Posted in General | 4 Comments

YouTube scalability Talk

I found this post on a talk about scalability at YouTube.

I knew that they were using Python, basic tools like ssh/rsync, mysql, and leased servers. What I didn’t get was some of the detail that the speaker revealed in this talk. The interesting bits are about MySQL and Memcached and how they followed the same old patterns as every other startup.

Everybody thinks MySQL and Linux are ‘super frantastic’. When you really have scaling issues, you will really get to see how production ready these systems are. The only reason these technologies are popular are because they are free, ship in easy to install distros, and regardless of their flaws… satisfy the requirements for 99% of the sites out there. I’m no longer a technology purist, I see the value in good enough. There is nothing wrong with being free and good enough. We use these very same technologies every day. It just annoyed me when I heard previous founders and CTOs exclaim that Linux and MySQL technology were so much better than ABC. They always got burned when their traffic started moving up the Alexa ranks. When they tried to stop the burning, they couldn’t find very many posts on the MySQL or Linux kernel lists to qualm their issues. When you are scaling, you are now in a rarified place. Keep that in mind as we move on.

Its also a widely held belief by many non-technical founders that these scaling issues are solved. Hey if ‘XYZ’ is so big and they are using this stuff, it must work (ie. you don’t know what you are doing). Company XYZ is hiring lots and lots of really smart people (RSP) to solve these sticky issues. They are spending days looking at single issues (why are we getting connection resets every ### connections?) They are staring at ‘netstat -s’ output. They are coming up with workarounds and hacks that ย they may or may not understand. They are looking at kernel, glibc, PHP, or Ruby source. They may now be customers of some large companies that offer ‘support’ for their hardware. They all have written software internally that solves their problems (I wonder how much of that is linked with GPL software? Ah, no matter, they don’t ship CDs ๐Ÿ™‚ This is now some of their technology or secret sauce… hacks. Some of the smarter of these companies are releasing this stuff into the open. Don’t hold your breath for Google releasing their gems (GFS, BigTable, etc.) to the open. They are smart, that really is their secret sauce.

Anecdotally, back in the day (eGroups 1999), we had a similar problem with paging as mentioned in the YouTube guy’s talk. For us, Solaris solved the paging for our Oracle databases. You could give priority to the pages backing executables so they wouldn’t get reclaimed by the VM system. Its interesting, yet not surprising, to see that it is still a problem these days. I think the presenter was surprised about this. The misconceptions about VMs by programmers, admins, and some OS people is a topic for another post. Anyways, I also had a tendency to turn off swap on machines. I like performance guarantees when I’m doing a high-performance production site. It is one thing to page out old ‘getty’ processes, it is another thing if your machine is constantly paging. Once the paging happens, it doesn’t gracefully degrade. Usually, there is so much expected performance from a machine that any consistent paging activity would start dropping the service level of a machine pretty drastically. Pagers will go off and MRTG graphs would start flying up. If you have a decent business, all of this will be ok since you will just do the quick fix of buying more hardware. Hardware is cheap compared to the $$ you can make on the internet.

The next interesting topic he mentioned was the need to re-invent replication, caching, etc on top of MySQL. Out of the box, MySQL is fine for that one big database in the middle. However, when you have to write software to manage the replication, and you are waiting on networks and disks, life can get a little white knuckled. Especially, if you are like everybody else and you write bugs. Bugs affecting a low level of your system (the persistence system), are very hard to diagnose and annoying to fix. Nobody likes to read the MySQL release notes to find out that the primary database (that you never take down, since it will take down the site) is running an old version that is slightly incompatible with the newer slaves, and hence not doing The Right Thing. This is detail oriented stuff. If you are doing this, you are in the danger zone. The presenter eventually describes what every team discovers at a certain point: we need a distributed database. A lot of sites just use a sharding technique, but this just leaves the door open for inconsistency and error. So far, nobody has solved this problem in a cheap and easy way.

One thing that has been solved in a cheap and easy way is caching. Memcached is the winner. It is easy to install, deploy, use and it is solid. Everybody uses it. I think they should add an M to LAMP for Memcached. It deserves it.

This also leads to another interesting artifact. I’ve often heard a few serial entrepreneurs exclaim that ‘man somebody needs to create a company that sells the XYZ that we all reinvent at every startup’. I’ve heard Fletcher say it and Skrenta has blogged about it. Usually this is some grid/distributed database solution. This is all good intent, but usually these same people (successful entrepreneurs) reinvent only when they have to (and they do). Competing with open source is difficult, and open source businesses are not shooting stars. Eventually they return back to their roots and focus on creating services on the internet. We’ll all let Steve Jobs create the nice shiny stuff. Fixing things would be nice, but it requires serious money and effort. The tools in this space will slowly evolve as open source solutions pop up out of the few companies at this level.

YouTube just proves that LAMMP is more than ‘good enough’ to scale. Yeah its annoying that you still have to hire really smart people. The future presentations from the future internet megaco will look very similar except there will be mention of the tool ABC which solves XYZ. Linux with Apache will still be a great combo. Regardless of the issues the presenter ran into, they still stuck to Linux, Apache, Lightty, and a few other open source bits. They didn’t have to do anything special other than buy into a CDN. The relative simplicity of these systems works. I say relative since it could be better. It is just relatively a lot better than trying to figure out the opaque magic of some of the other systems of the past (think older Solaris, Windows, etc.).

So, what am I trying to say:

  • Overall, the same mistakes and technologies will be used since the established practices for ‘really large scale’ are not written or understood. Only off-the-shelf or ‘duh’ patterns will be common. (“Duh, install memcached dude”). People will continue to come up with new ways to reinvent replication for MySQL.
  • Ironically, for the number of users these sites have, they don’t spend any money on software. Its hard for the baker to buy bread from somebody else. The free/open source software continues to be better and better at solving the issues. You can’t beat the price. I doubt I will see many commercial companies succeed at monetizing this space.
  • People are not efficiently utilizing their grid. Google is probably the exception here. Very few people are skilled enough to adequately understand the computation of their application and tune their Disk, TCP, etc. adequately to achieve the performance levels that could be possible. They usually fly on the defaults, which on existing systems, gets them pretty far. …which leads to the next point…
  • Really Smart People will still be required, but the hardware and software will progress faster at scaling… if the current workloads stay nearly the same. Said another way, systems in the near future will handle 4x? the users that the existing systems do… at the same cost. You may see more complex sites get a little farther along without as many Really Smart People.

BTW, apologies for the long post. It’s just an interesting topic.

Posted in General | Comments Off on YouTube scalability Talk

One of these days… I have to really dig into Lisp

After reading this thought provoking post on Common Lisp, it really does drive home that I should learn Common Lisp when I have some copious spare time :-b. Unfortunately, it seems that the quickest way to learn is to watch and interact with a Lisp master. They always seem to have these special tricked out environments for Lisp hacking.

Posted in General | 1 Comment

One Year!

About 1 year ago, Tod had an idea, and it became …

postroller logo

3 months after that, we rebranded and we became …

BrightRoll Logo

It’s been 1 year and a a worthwhile endeavor. The whole team has learned so much about online video! We celebrated the occasion last Friday and reminisced a little. Monday is about to roll around again, and there are still a lot of interesting deals to be done. Who knows what will the next year bring… but if the last year is any indication, we are ready!

Posted in General | Comments Off on One Year!

I wish I had this about 5 years ago

Here is a neat library that I wish was around 5 years ago:

templatemaker

Here is the blog announcement for the project. Go check it out.

It’s only for python, but I suspect you’ll see something similar to this show up for every major language pretty soon. In fact, in the blog comments they even mention a similar project in Ruby called Ariel that was sponsored by Google’s Summer of code.

Posted in General | Comments Off on I wish I had this about 5 years ago