Thanks to Robert, we now are pulling in data from our sources every 4 hours. Previously we were updating the data only once a day (at 3am). This was due to the fact that pulling in the data tends to tax the server, and thus we didn’t want to run it during the day. Robert has changed the import process though, to launch a new EC2 instance to run the imports, and shut it down when its done. So very very cool. This allows us to run the imports without taxing the server, and also save money because the “import instance” is only up and running for a few minutes a few times a day. Oh man do we love EC2.

On a side note, we have temporarily disabled the “Restaurants” section of BetterOmaha.com. We’re currently gathering and improving the data (including taking pictures of every restaurant in town), and as such we have decided to take it down for a bit. Stay tuned because the new version of the BetterOmaha Restaurants Guide will be the BEST resource available…

After a few days of wrangling, data-importing, and pulling our hair out, we’ve got BetterOmaha.com back up and running (mostly). Everything related to the news is working (search, imports, etc). The Restaurants section still has some corrupt data, which will will be correcting later today. Additionally, we’re missing news imports from the last few days, so those will slowly be added today as well.

All in all, we had nearly two days of downtime, for which we sincerely appologize. As a beta product, we’re still working through some of the technical details regarding our hosting. We thought we had it a little more figured out than we did. Sorry.

Some of the root causes we know, and some of them are still unclear. We will be performing some forensics on the incident, and will provide a full writeup here, likely later this week.

Thanks for your patience and continued support…

The BetterOmaha Team

The BetterOmaha.com site is currently experiencing issues. We are aware of the problem, and are working hard to fix the problem, and get the site back up and running. We will keep you posted with status updates here, and will also provide a full explaination once we know more.

Thanks for your support and understanding.

UPDATE - 4:40pm CST - Now redirecting all traffic going to betteromaha.com, to this blog. We are unable to get the original ec2 instance rebooted, and are currently in the process of rebuilding a new image. Stay tuned.

UPDATE - 6:10pm CST - We have the new instance up and running, and we’re currently in the process of getting the 15gig BetterOmaha.com database restored.. It will take a while. Thanks for your patience.

UPDATE - 10/5/08 2:30pm CST - Yes, still down. The database restore tasks provided by EC2onRails are timing out on our size of database. We’ve tried several times to get them to run. We’re now resorting to a manual reload of the database from the latest backups. Should have something soon.

UPDATE - 10/6/08 2:00am CST - Unable to get the server restart issue resolved. It appears to be a rather nasty issue with Amazon EC2, and Ubuntu 8.04, specifically AMI’s that are generated off of http://alestic.com scripts (as ours are). We are currently in the process of merely getting the server up, and we will worry about the fact that the server can’t be rebooted, sometime in the future.