Code

Last night I was playing Star Trek Online for the first time in a few days. I played a good couple of hours before the game took a dump on me. After they got over the initial hurdles of the launch, the game has been relatively smooth to play so I've been happy. Plus the problem occurred during a load from one zone to another after a quest was completed, so I didn't "lose" anything. I got a message saying that I was "Disconnected From Server", but it doesn't give me any options to do anything about it (like quit, or reconnect). Being the good little QA minion that I am, I like to report these type of issues to the developers so they can fix them. I knew there was a tool that came with STO for this exact purpose. It wasn't widely published, that I know of, but I ran across the EXE once. So I dug into the folder, found and launched "crypticError.exe" and then it started to crumble.

The first screen is fairly simple, you select the running process name and explain what you did/what went wrong. Simple enough; I did so then it started collecting data. Great, after about 10 minutes I came back to find it still uploading a dump. I thought this was a little odd so I looked closer. The "dump" was over 1 GIGABYTE in size, which corresponded directly to the exact size in RAM the game was taking up. It was uploading the ENTIRE game's RAM state. Now, don't get me wrong, I understand how that can be useful... but this is a VIDEO GAME on a CONSUMER'S machine. You launch the thing and it is taking up over 300MB of RAM. Even on my go go speedy DSL line, it was going to take about an hour and a half to upload the entire dump. My DSL is also fast, and uncapped. What about those people who have 1.5meg lines, and have bandwidth limits.

Seeing things like this just blow my mind. It wasn't like I clicked some checkbox that said "Upload over 1GB of RAM state", because you don't have any checkboxes to hit. Plus, I hate to say this, but there is no damn reason for the error reporter to upload the entire RAM state. Why does one need to upload all the texture data? Report the file names loaded into memory and the checksums. If a checksum doesn't match, then that could have been the culprit.

It is safe to say that shitty design like this is why some error reporting tools never get used. I wouldn't be surprised if the devs were actually shocked the first time anyone let this tool run it's full course (provided anyone has been that dumb).


<?php
$dir = "./mydirectory/";

if ($handle = opendir($dir)) {
  while (false !== ($file = readdir($handle))) {
    if (preg_match("/^\..*/",$file,$junk) || preg_match("/^BAD.*/",$file,$junk) || preg_match("/^TEST.*/",$file,junk){
      //Skip it
      print "Skip: $file\n";
    }else{
      //We want to edit this file
      print "--EDITING--: $file\n";
    }
  }
  closedir($handle);
}
?>

This isn't exactly the most complicated piece of code, but it was extremely annoying when I got the error "Only variables can be passed by reference" back from PHP. Getting this error really made me miss Perl again. Perl's interpreter had the most fantastic error catching. It would (generally) tell you what the error was in language that was very easy to understand, and in some cases even suggest the fix (Like you forgot a semicolon on line X). PHP on the other hand, loves these damned messages that make no sense.

So what's wrong here? The preg_match line, the final "$junk" is actually just "junk". I forgot the string symbol ($). Very simple issue, very easy to fix... but very difficult to find information on. Normally a quick google reveals all, but for this error... Nada. So if you happen to google this error and land here, hopefully this solves your issue.

I've been doing some reasonable amount of programming as of late, working for miscellaneous projects around the Wikimedia universe. I'll announce the specific projects as I get things to a "finished" or releasable point. For now I thought I'd tease you with random bits of "fun" I encounter when working with PHP.

Today's posting is not the failure of PHP to live up to my expectations, it is about HTTP Error 417 "Expectation Failed". I was writing a file upload bot to push images up to Mediawiki sites, and on my local testing environment it worked perfectly. I then re-pointed the script to upload to Wikimedia Commons and it went all to hell. I started getting HTTP Error 417, which for as long as I've been around the net, I had never before seen. I turned on the debug info for curl:
    curl_setopt($ch, CURLOPT_VERBOSE, TRUE);

I watched the headers closely and saw that my application was sending "Expect: 100-continue" and after a bit of googling I found that it was something from the HTTP/1.1 spec on negotiating data upload. This is great, but my application sends that it is playing in HTTP/1.0. Obviously this isn't a good thing, and it has been known about for a while. Why it still does it, beats me, but there is an easy way to "fix" it in a PHP application:
    curl_setopt($ch, CURLOPT_HTTPHEADER, array( 'Expect:' ) );

That's it.
This way curl will send an empty "Expect:" instead of the full "Expect: 100-continue", and subsequently it will get ignored by the server you're talking to. I've found that Apache handles this sort of issue gracefully, which is why my local tests worked. The issue is Squid (which the Wikimedia Foundation uses as reverse proxies) which is much more of a stickler for standards compliance.

Over the years as I've spent more time on my cell phone for data access (markedly more since I got a company phone), I've wondered why more websites don't have a mobile version. Of course there are a few big companies that have mobile versions (like Google) that are basically as good as their regular site (granted Google's regular website isn't terribly complex). Some companies (such as Starbucks) have mobile sites, but the functionality is reduced heavily from their "big web" sites. And then there is most other sites, simply have no mobile version at all. This is extremely frustrating when you are forced to do regular work on a mobile browser.

Recently people have been trying to build the "rich mobile web", bringing all sorts of cool features that you find on the "big web" down to the mobile web. This is of course a fantastic idea as the direction we are headed is that in which everyone has a cell phone with a data plan. The Japanese are way ahead of most of the world in this respect with Mobile TV, QR Codes, and all sorts of other cool stuff. That fantastic idea aside, we (in the United States) simply don't have the bandwidth (yet) to have a rich mobile web. In fact just visiting a normal website on a mobile phone can leave you hanging for a minute or more while all the miscellaneous media is downloaded. It is a truly miserable experience.

So to the question of the day (AKA the post topic) is: Why don't more people build mobile sites?
Answer: It is a freaking pain in the ass.
This I am finding out from personal experience. I recently started making our company's website (one of them) mobile compatible. The site itself is extremely simple, only a handful of pages. I had recently converted it to use PHP for a simple template system. I figure it would be easy to change the header & footer for mobile browsers, dump some images and "Voila! Mobile compatible website". Yea, not nearly that easy. HTML Transitional 4.01 (which I did the site in) is not "Standards compliant" for the mobile web, so I am supposed to use the XHTML-MP 1.2 profile. Well, obviously, XHTML != HTML so the entire site is very standards un-compliant on the mobile (according to the W3C Validator, but it works in IEMobile & Safari Mobile). I spent most of the day working on the site and got it from a 1 or "bad" to a 4 or "good" according to Ready.mobi (a mobile-readiness evaluator). An entire day for a handful of pages to be "good" and that doesn't even take into account the TONS of errors the W3C has with my hack.

I'm continuing my fight, but so far it seems like the only real option a person has when building a mobile website, is to build a completely separate site. Either that or build a site in a middle-ware that will convert it to the appropriate version, which seems even more complicated (but easier to maintain). Not fun. But it does explain why the mobile web is not a popular web.

I've never been one to use mysqli in PHP, I've never needed to. Recently though, I've started using stored procedures, which necessitated the move to mysqli. Now, this really isn't that big of a deal. After all, a search and replace of mysql_ with mysqli_ and adding the connection flag, and you are good. Here's where it got fun for me though. I was using Custom Session Handlers (so I could write to the DB, instead of to a file - like many people do). Everything worked peachy, except for my "Write" override. For some god damned reason the thing kept kicking out Warning: mysqli_query() expects parameter 1 to be mysqli, null given.

I spent 3+ hours trying to debug this problem. I thought there was something in my code stomping the connection variable, or otherwise munging it (I was reusing old code). I put in DB "Ping" checks EVERYWHERE. They all worked, except for that subroutine. Well, after the aforementioned frustrating hours, I seriously got into the googling because someone had to have some insight into it. I found some bug reports and other comments that eventually steered me to a forum post.

Short answer? Add session_write_close(); to the end of the script. For what ever reason the session shutdown is "Broken" and tries to shutdown after the database is disconnected. So shut it down early and you are good. Yea. One simple call and it fixed my problems. ::Grumbles::

I've been working on a PayPal IPN Script. The key to PayPal IPN is that they post the data to your server, you have to post it back to them with one extra query attached. Most scripts I've seen read STDIN directly to get the post and then break out the rest of the data from there, but I dont want to do that. I like my mod_CGI. The only problem is that when using this module, you can no longer directly read the POST data. I've found a work around

@names = $q->param;foreach $key (@names){$z .= $key . "=" . param($key) . "\&";}

This snip of code will rebuild the POST in the order it was recived into the variable $z. Its not a "preety" solution, but it does work (It also adds an additional & at the end, but thats ok with my needs since I have to attach another query to the end for paypal).

Otherwise I have to say working with the PayPal IPN system is very intresting, and once you hack through all the crap thats thrown at you - its really very simple.

So Asterisk Manager is a telnet service that allows you to control many aspects of Asterisk over telnet. Yea! Now getting it to what you want using Net::Telnet is aparently a different story all together. I spent a number of frustrating hours working on this today, but I can happily say that I got it working. I'm trying to setup something resembling a Call Wake Up service - Not nessarily to sell to the public, but for my own amusement (and to help me wakeup). Right now I have some half assed perl script all hacked up and hooked into crontab on my debian server in the office. If it works right for at least a few days, I'll figure out if it can be expanded and hooked into MySQL etc (mysql must be used for all projects, big and small).