Unweb: A Postmortem
With some time between projects, I began to experiment with the <CFHTTP> tag, which allows ColdFusion to make an internal http request to a web server and process the results as a variable. I wondered how easy it would be to parse the data that was returned from an http request and generate loose "statistics" off what was found, such as font tags instead of cascading style sheets, or tables that weren't properly closed. In a sense, I attempted to see if I could discover errors in other people's HTML.
My initial tests proved successful, and I was able to generate basic statistics from these parses. In order to keep my interest, however, I had to make the project fun, so I turned it into a game. The idea was to award points based on good html, and to take points away, if bad html was discovered. When the score was tallied, the web site was given a rank. At that point, I decided it might be fun to use this rank as a "health" or HP value. I then began playing with various other factors and came up with an Attack Rating and Defense Rating for the website (how hard it attacked, and how well it defended). One thing lead to another and I soon had a player concept as well, allowing a person to type in a URL, parse the HTML, and fight the "monster", with a series of clicks to a "Fight" submit button.
Thus, Unweb was born. The game grew and went through many evolutionary revisions, adding support for multiple items, new item types, ladder rankings, and more. Inevitably, Unweb sufferred through several moves to various webservers itself, where a combination of poor programming and detatched administration led to its eventual downfall. For as long as Unweb was alive, however, it seemed to offer a bit of a distraction in the form of some fun and vengeful tactics for the wary adventurer who had a few minutes to kill, and a website that needed to be beaten on.
What Went Right
In developing Unweb into a full-fledged webgame, I think that I had a number of ideas that turned out rather well in the finished game. Whether or not this was a result of careful planning, cunning ingenuity, or pure dumb luck has yet to be determined.
- Parsing Meta-Tags for Random Item Names: One of my earliest ideas when I began modelling Unweb around the whole "Diablo" theme was to have items dynamically generated after a website was defeated. Parsing Meta-Tags out of HTML can often reveal interesting tidbits about sites that are commonly used by Search Engines as index data. My thought was to parse these keywords out and turn them into the names of the items, to add a bit of flavor, humor, and randomness to the game. I figured that if a player tried Unweb out on his or her own website, and was awarded a weapon that a family name, or some other piece of random info it it, the player would be amused enough to try again. This seemed to be a successful tactic, since players responded as finding that one of the most interesting (and funny) aspects to the game.
- Allowing Players to Sell Items To Each Other: When Unweb went into early beta, I approached a few programmers at Macromedia (then Allaire) about their opinions on the game, whether or not they thought it was fun, and how I could improve any of the CFML code behind it. One suggestion came in from Raymond Camden, author of the famous "Death Clock". Since items were dropping in the game, as well as gold, he argued why I wasn't supporting the ability for users to sell items to one another. My initial reaction was, how on earth would someone build support for two users selling items to each other, that aren't necessarily online at the same time. Ray responded with simply, "Allow the player to put the items up for sale, as a fixed price, and allow other players to browse the for-sale items at their leisure, picking up items that they want (if they have the gold)." Duh! It couldn't have been simpler. Adding in support for items being sold back and forth between players proved massively successful, especially with "collection factor" of the set items.
- Implementing a Diablo II-style Ladder: Ignoring the fact that Unweb was a game, it was at heart, simply a website, which meant many users would be accessing it at once. So, in order to give the players a sense of "purpose" to the game, I opted to add the competitive factor of a Ladder, much like what is used in Diablo's Battle.Net service. This also proved to be a great addition to the game, since it not only encouraged users to continue to play more (to vie for the top-most position), it caused them to bring in their friends, who in turn invited more and more people, in order to compete.
- Optimizing Logic in <CFSCRIPT>: The brunt of the game logic fell within each attack round. This is where all the bonuses, items, armor, special attacks and more were thrown together into a damage calculation and applied to the website being hit, and the player doing the attacking. In order to get the absolute maximum amount of performance out of CFML, I ran through several iterations of optimizing the code, which inevitably led to doing all the calculations in <CFSCRIPT> tags. Moving from my initial code to the optimzed cfscript shaved several seconds the final product, which ultimately all adds up when hundreds of users are hitting the site.
- Moving the HTTP Fetch/Parse to a COM Object: It became clear early on that <CFHTTP> was not going to be robust enough to handle hit after hit in a game of Unweb. Thanks to a fellow friend and programmer, Dave Sausville, I decided to move the logic of the HTTP fetching and parsing of the web result into an externally compiled C++ DLL, in the form of a COM object. As you can imagine, this improved the speed and stabiliy of Unweb immensely, and was probably the single best decision throughout Unweb's development.
- "Pimping" the Website out to Diablo Fans: Once Unweb finally went live for its initial run, I immediately began emailing website admins about it. The goal was to get the word out and hope that it would spread, based off the user experience. The most successful clique of this fanbase ended up being fans of the original Diablo II game. I made sure to contact the admins of planetdiablo.com and diabloii.net, and as well, hung out in #diabloii on IRC in order to drudge up some more users. This, in my mind, was a success, simply because they went in understanding the basics of the game, and getting the "inside humor" that gamers not familiar with Diablo missed. It was probably their word-of-mouth that led to Unweb's userbase eventually growing to 15,000+ users.
What Went Wrong
- Putting Too Much Faith in <CFHTTP>: Since I was initially only "playing" with the idea of posting out to a user-specified website and parsing the result, <CFHTTP> seemed to be the most natural solution (since I was using CFML). However, I should have trusted my gut instincts more and decided on using a different technology sooner, since it is a well-known fact in ColdFusion circles that <CFHTTP> isn't dependible and doesn't do well under load. Alas, it was the first giant bottleneck of Unweb when the game went live, and cause the servers that hosted it to buckle and completely shut down at times. I did eventually move to an external COM object (see above), but in all honestly, I should have avoided <CFHTTP> like the plague from the very beginning.
- Overestimating CPU Usage: Unweb was a very CPU-itensive web application. In retrospect, I really overestimated what the web server could handle when it boiled down to sheer number crunching, which is exactly what Unweb was, at heart. Everytime a player clicked the "attack" button, the calculation of all the bonuses, all the damage, all the defense and so on, had to be re-crunched. Sounds somewhat pitiful, when you think about it (isn't number-crunching what programs are supposed to do?), but Unweb was written in CFML, which is a scripting language for web applications. Most games, however, benefit from compiled binary objects, that are optimized for speed. Since I had never implemented a game in this fashion before, I really had no point-of-reference to gauge what kind of performance I would get. Alas, the real test came when Unweb got beaten on by players, and if I knew then what I knew now, I wouldn't have written Unweb in CFML at all.
- Underestimating Cheaters
- Ignoring SQL Server Indexes: This is your typical slap-me-I'm-stupid reason. There is absolutely no excuse for not having indexes on a database. Unless, of course, you're a moron and forget, as was the case with Unweb. With focusing some much on the actual fight algorith, creating random item drops, building a store, supporting a ladder, and so on...I forgot one of the most basic optimizations every database should have. For those of you unfamiliar with Database design, an "Index" is much like an index in a book; it allows the database to "look up and jump to" specific data very quickly, as opposed to scanning page after page after page of data until the required info is found. All good DBAs will tell you a properly designed database will be indexed appropriately. On Unweb's release day, its database wasn't. That's a bad thing. And yes, I have been spanked for my behavior.
- Botching the Authentication Process: Let's be frank: I specialize in back-end application logic development. I almost never touch the UI (user-interface), save to drop a few form fields into a webpage (and I prefer it that way). Most of the UI work done at my office is done by a team of experts that specialize in creating and optimizing UIs to suit the needs of our various clients. So, when it came time for me to put a front-end on Unweb to allow players to actually "play" (who would've thought!!), I missed the boat big-time. For starters, if you want to entice anonymous people to your site, to "see what all the commotion is about", the first thing you should NOT present them with is a login/password window. I would have to say 75% of my initial traffice started...and ended...on the login page. Creating a new account should have been much more intuitive, and anonymous users should've been allowed in for a few "free" rounds, first.
- Not Having a Dedicated Webserver: About the time that Unweb was getting ready to go live for the first time, I had a number of options for hosting. I had administrative control of a dual PII-400 w/256 mb of RAM that was already acting as a webserver, with ColdFusion 5 installed. However, it was also being shared by a group of friends to do some "online gaming" in the evenings. Once I saw how the initial load of users spiked the processors (see point 2), I knew that idea was out. I had a friend running a webserver on a Duron 700 w/768 mb of RAM, so in theory, that made a bit more sense. Theory, unfortuantely, isn't always close to reality. That box was constantly coming down after Unweb was hosted there, no doubt related to point 1. After a short stint there, I was approached by a fairly large online gaming conglomerate that offered to host my site for free on one of their uber-boxes, a dual PIII-900 w/ 1GB of ram and a "fat pipe" (about a T3) worth of bandwidth. However, they backed out, once they discovered the game was written in ColdFusion, and since they were mainly an ASP house, weren't willing to fork out for the hefty price tag of a CF license. Finally, I moved the site to an inexpensive CF-geared host down in Texas. Unweb has the least amount of problems here, but alas...it was still causing CFServer to crash from time-to-time, which is simply not acceptible on a production host. Since Unweb was moved from home to home more than an orphaned child, it was doomed from the get-go.
- Not Implementing a Caching Mechanism for Users: One of the best optimizations that Unweb never got was a "timed" commit of all user-specific vars to the database. Originally, Unweb would commit all the user's data to the database at the end of every fight. Well, some fights could last mere fractions of a second (nevermind point 3). All these extra commits to the database could've really lightened the load of Unweb, if they had been simply stored their data in memory, and commited changes-only after a given time period. Coincidentally, I began to design a "sequel" to Unweb that subsequently fell to the wayside, but one of its features was exactly this: a timed mechanism for commiting data. The performance really was improved and I honestly believe that Unweb could have benefitted greatly from this simply optimization.
: Point 3 kind of goes hand-in-hand with point 2, simply because I knew that with many online games come cheaters. However, I honestly had no idea Unweb would become as popular as it did, so I was never thinking about cheaters as having a significant impact on anything. If I found out someone was cheating, "big deal", I thought. So they get to the top of the ladder in one day, or figure out how to get a specific item to drop, or maybe to decode the "randomness" of the gambling. All of these issues paled in comparsion to what cheating players ended up doing, which was causing the actual server to buckle. They did this by writing "scripts" which automatically played Unweb for them; logging in, selecting a website to fight, and the actual clicking of the "attack" button...all done with some clever code. As clever as it was, these "scripts" acted more like load-testing tools, smashing the server with hundreds upon thousands of "hits" every minute. I managed to squeeze in some last-minute anti-cheating ploys, but to this day, I still get the occaisional email from someone who managed to get around them. Note to self: If you make an online game... factor cheating in. Period.
All in all, Unweb was an exciting adventure to undertake. I have built small games in the past (and I intend to build more in the future) but I think Unweb struct a chord with many players. Not only was it a total rip-off of Diablo's style, but the idea of fighting websites was a real "kick" and I truly believe players that hit it hard, did so because they got some amusement out of it. I still receive emails on a regular basis on "whatever happened to that web-site fighting thing?" and "is Unweb hosted anywhere?" (I even get requests for source code!), and all of these I appreciate, since I am still amazed at the response it garnished. The best response I think I ever receieved was from a professional game-developer, who considered me one for Unweb's creation. To that, I am truly honored. Perhaps one day in the future Unweb may live again in some other form, but for now, Unweb has fought its last website...
...or has it?