a timocracy of one

I’ve spent plenty of time kvetching about problems running my Typo blog on Dreamhost. I acknowledge, though, that despite some of DH’s other hiccups, the core problem is that running Rails, on Apache fcgi, on a shared host is a Bad Idea. And while I can handle the initial set-up of a slice/VPS I don’t want to deal with the maintenance. And I don’t want to pay an arm and a leg for a fully managed set-up, just to host small sites like blog.

So when my site started going down constantly, again, due to processing getting killed for being over the memory limit, most likely, I decided I would try out Dreamhost’s new “Private Server” offering, which is sort of a managed pseudo-VPS: cheaper than a true VPS, no root, they manage everything, but you do get virtualized/dedicated resources. Sounds perfect for what I need.

Unfortunately, there is a waiting list… and my site is down now! So I decide to email support and say, hey, I’ve sent some business their way over the years, my site is down now, any way to bump me to the front of the list. Well they did – but apparently side-stepping their usual processes. I show as still wait-listed in their web, panel, but all the sites go down on my account, all of a sudden, from the switch. It turns out they have moved me over to the PS, but the DNS didn’t get updated correspondingly for all my domains.

After a day or so of wrangling with their support queue, Jason got things fixed and it seems to be working. Now I just need to see how well it works for my needs. I’m going to just stick with the Apache fcgi, for now, because it works, I’m curious how the fcgi performs when not constantly being killed for exceeding the shared allowance of memory, and Dreamhost’s proxy set-up proxies you to a single mongrel by default, so I’d need to set-up up something like nginx to proxy to my mongrel clusters, and then keep all those up myself. Not that big a deal, but I am going to run with the out of the box experience a bit first, and see how little admin work I can do.

Some useful info on the DH wiki for git and on kernal.org’s guide to setting up git over http. The latter is especially helpful for debugging suggestions (like using curl to check your .netrc access permission). Using the info in these guides:

• Create a repo, or use an existing one
• Enable WebDav access on a DH directory, using their panel. Set-up the username password for the committers, and disable the password protection box for read-only anonymous access (might be easier to keep it on until you have everything set-up right)
• Create a .netrc with the auth info for this webdav
• Test your access using the curl command in the kernal.org guide (you may find it useful to have password protection enabled at this point,).
• Connect to the webdav share using the username and password.
• Copy over the .git folder contents into the root of the webdav share (this is key, because DH’s anonymous access won’t show . files)
• cd into the webdav share from your local machine and run “git-update-server-info”
• profit

You should now have access to http://yourserver/gitfoldername and http://gituser@yourserver/gitfoldername. For example: you can “git clone http://blerb.infosauce.org/git/” for read-only access to the blerb repo. Now, if I can only get gitweb working despite the limited access control imposed by DH webDAV.

Update: Changes the url for blerb to still work right after I changed it around while playing with gitweb. Also, you should use git://42squared.com/git/blerb if you actually want the real blerb repository.

So I’ve got some spam that is being submitted through AJAX (I have non-Ajax comments turned off) and slipping through akimset. One could think maybe it’s being hand-crafted, because it sometimes makes sense on a tech blog, but it isn’t pertinent to the actual specific article. I guess they know use a database of tech comments, maybe scraped off from existing comments, and apply them willy nilly on blogs that cover these topics, while setting the url field for the name to their spammy site? Why don’t I think it’s hand-crafted?

If LogoMaid made your logo, you are supporting trademark theft and copyright violation has a comment “why it’s always writing NULL in empty fields?” and I forgot how much I really hate shell scripting has “Predicted values in template editing can be done through any PHP script?”. So they sorta make sense, just not as comments on these articles, or even on any of mine I can think of. Database of techy comments for use in spam?

Neither Analytics or Feedburner have directions for attaching their scripts “properly:” in the head element and without in-lined javascript. Yet both of them work fine, when you do.

For feedburner, just follow the directions for the simplestats, but put the script tag in the head, not in the body, as directed:

<head>
  <script src="http://feeds.feedburner.com/~s/yourusername.js" type="text/javascript"></script>
</head>

For Analytics, attach the script, and use some method of calling a cross-browser DOMContentLoaded. I, use the one from lowpro since I’m mainly developing in Rails, so using Prototype is a no-brainer.

<head>
  <script src="http://www.google-analytics.com/urchin.js" type="text/javascript"></script>
  <script type="text/javascript">
    _uacct = 'UA-yoururchinaccountcode-1';
    Event.onReady(urchinTracker);  
  </script>
</head>

While properly setting up my Rails app server maintenance tasks, I noticed that my Typo install was running even more bloated than I remembered – no wonder the Dreamhost watcher processes kill it off. After digging around and trying a couple things it seemed that deleting your unused text-filters, sidebars, and plugins does nicely, for trimming down Typo.

Well, I got my Typo install down from running in the 50s to 30s (megs)upon the initial start (it still grows as it leaks, of course). See the glory of what I hacked out of the vendor folder.

!      vendor/bluecloth
!      vendor/plugins/magnolia_sidebar
!      vendor/plugins/haml
!      vendor/plugins/aimpresence_sidebar
!      vendor/plugins/fortythree_sidebar
!      vendor/plugins/technorati_sidebar
!      vendor/plugins/typo_textfilter_tmcode
!      vendor/plugins/tada_sidebar
!      vendor/plugins/typo_textfilter_lightbox
!      vendor/plugins/typo_textfilter_markdown
!      vendor/plugins/tiny_mce
!      vendor/plugins/xbox_sidebar
!      vendor/plugins/typo_textfilter_amazon
!      vendor/plugins/typo_textfilter_sparkline
!      vendor/plugins/amazon_sidebar
!      vendor/plugins/recent_comments_sidebar
!      vendor/plugins/typo_textfilter_textile_and_markdown
!      vendor/plugins/audioscrobbler_sidebar
!      vendor/plugins/backpack_sidebar
!      vendor/plugins/xml_sidebar
!      vendor/plugins/typo_textfilter_flickr
!      vendor/plugins/typo_textfilter_code
!      vendor/plugins/flickr_sidebar
!      vendor/plugins/typo_textfilter_smartypants
!      vendor/plugins/upcoming_sidebar
!      vendor/plugins/sitealizer
!      vendor/plugins/fortythreeplaces_sidebar
!      vendor/rubypants
!      vendor/flickr
!      vendor/jabber4r
!      vendor/sparklines

Here are the corresponding environment.rb changes. I also deleted all but the Textile and none filters in the admin interface. The first thing I tried chopping out, though, was the newly added sitealizer. Since this is run via an around filter, you have to edit your application.rb thusly, if you take it that far. I, also, got carried away and took out BlueCloth, since Textile is handled by RedCloth. In doing so, I found at least one place in the admin interface were it is hard-coded, and will cause your new content form to choke: app/views/admin/content/_form.rhtml

UPDATE – five minutes later: I forgot, I also turned off live Preview to stop hammering my poor app, and started editing directly in Textmate, from the textarea

UPDATE – 4/19/2007 – Found another hard-coded instance of BlueCloth that I had to comment out: app/models/text_filter.rb/comment_help line 96

f.help_text.blank? ? '' : help_text  # "#{BlueCloth.new(f.help_text).to_html}\n" 

crontab

MAILTO=me@mydomain.com
# minute (0-59),
# |      hour (0-23),
# |      |       day of the month (1-31),
# |      |       |       month of the year (1-12),
# |      |       |       |       day of the week
# |      |       |       |       |
  */5    *       *       *       *   wget -q -O /dev/null http://mydomain.com >/dev/null
  @hourly /home/myuser/clear_timocracy_sessions.sh
  @daily /home/myuser/rotate_timocracy_logs.sh

Grabs the site every 5 minutes to keep the fcgi alive on Dreamhost, and calls scripts that clear the sessions and rotate the logs h3. clear_timocracy_sessions.sh

#!/bin/bash
/home/mysuser/mydomain/script/runner -e production "CGI::Session::ActiveRecordStore::Session.destroy_all( ['updated_at <?', 1.day.ago])" 

Clears out sessions older than a day. h3. rotate_timocracy_logs.sh

#!/bin/bash
cd /home/myuser/mydomain
/usr/sbin/logrotate -s log/logrotate.status config/logrotate.conf -f; touch public/dispatch.fcgi

Calls logrotate and then updates the timestamp on dispatch.fcgi, so Apache will create new worker processes h3. config/logrotate.conf

"log/*.log" {
  compress
  weekly
  delaycompress
  missingok
  notifempty
  rotate 52
  create
}

Archive weekly, keeping a years worth of logs. delaycompress – wait to gzip them, till the next week, both for ease of reading the most recent and so there is time for the fcgi threads to get killed off, since they are pointing at the old log.

After an email exchange with Dreamhost I have had it confirmed that their watcher processes monitor on a per user basis. This means that if you run each app under a separate user you are less likely to have problems with getting your fcgi processes killed.

Excerpt of my email exchange with Dreamhost

> Is it true that the processes are watched on a per user basis
> as in running each app under a different user name will
> mitigate problems?

Yep, that’s correct!

It seems like logical enough behavior – you just want what was entered into the textarea to be represented in the most logical manner in the markup.

This:

A paragraph Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean suscipit metus vitae magna. Phasellus euismod ultrices justo.

Another paragraph Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean suscipit metus vitae magna. Phasellus euismod ultrices justo.

This paragraph
has linebreaks
though

to become this:

<p>A paragraph...</p>
<p>Another paragraph...</p>
<p>This paragraph<br/>
has linebreaks<br/>
though</p>

Unfortunately, this isn’t how it works in RedCloth 3. These are your two options

without :hardbreaks on

<p>A paragraph...</p>
<p>Another paragraph...</p>
<p>This paragraph
has linebreaks
though</p>

with hardbreaks on

<p>A paragraph...<br>
Another paragraph...<br>
This paragraph<br>
has linebreaks<br>
though<br>
</p>

Which means there is no way to get that right spacing (multiple returns where ignored in my testing), much less the markup you want: paragraphs with possibility of linebreaks in them.

The only way I found around this was by installing the latest RedCloth 2. To avoid problems with this being superseded by a later version of RC3 (and to get around not being able to install gems) being installed on the host, you can drop it in vendor and specify a require with a version in environment.rb. If even this won’t work on your shared host, take out the require and just move the redcloth.rb to the root of the vendor folder and Bob’s yer uncle.

Update 04/10/2007

I found this in the Collaboa environment.rb. I haven’t tested it, myself, but it looks like it should work nicely for forcing the version you want.

# We want features from RedCloth trunk (> 3.0.4), so unload the RedCloth gem if any, and re-require
Object.send(:remove_const, :RedCloth) if Object.const_defined?(:RedCloth)
require "#{RAILS_ROOT}/vendor/redcloth/lib/redcloth" 

As you can no doubt see, this blog has return from its hiatus. As I’ve complained about before, running Typo on Dreamhost isn’t that dependable, so while I was busy wrapping up the sale of my condo and then moving to Tucson, I let it slide. But now that I am somewhat settled in, I’ve gotten around to the updating, tweaking, and praying required to get everything working again, and will probably be keeping a little closer eye on it.

I’ve turned on various spam protection measures (such as Akismet, disabling non-ajax commenting. moderation, etc), and am hoping that will alleviate some of the performance issues. If not I am thinking of moving onto a new system- looking for something that is less of a pain for me to maintain with this set-up, since I will be needing to get a job, and thus will have better things to do with my time than getting frustrated with Typo/Dreamhost-fcgi. I’ve been considering the newest kid on the block in the Rails blogging world SimpleLog, but maybe things will stay functional enough that I don’t have to go down that road and spend more time converting everything over.

Either way, I’ll be posting again here some, as I work on finally getting the Lost River Outfitter’s site updated with the new functionality that I promised them as a Christmas present ages ago, and then finally get around to playing with some of my ideas for InfoSauce 2.0. If I get a non-tech job (like my plan to work in a bike shop), I’ll probably be willing to spend more of my spare time developing, but either way I’ll try and post semi-regularly as I come across cool stuff.

As well, I’ll probably be working on getting the design updated with Stf’s work (after I feedback to her on the first round logo stuff), so I can get rid of the stupid placeholder header image, fixing the homepage nav in Safari, putting my resume up, and other house-keeping as I have time.

A while ago I gave mt’s new GridServer a whirl. I was looking to jump the DreamHost ship for my primary hosting, especially the Rails stuff, and MediaTemple had this new cool sounding GridServer at a quite reasonable price – $20 bucks a month. At that price-point they are positioning themselves between cheap shared hosting and VPS/dedicated services, and I was intrigued. The potential for gradually scaling up, and the potential reliability gain were tempting.

First off, though, it’s not a Grid, technically. It’s more of a cluster, and clustered computing is not exactly groundbreaking. The only ground being broken here is a VPS alternative at $20/month.

And then you get to the fact that it’s hard to tell exactly what is Gridified. By reading around the best I could tell is only the web-stack. Well, as it turns out, most of my reliability issues with DH have not been because of the webserver crashing: it’s been router issues, or problems with the DB servers, or general datacenter issues. Well, these are all things that are not resolved by “the Grid.” Either your host already handles them well, or it doesn’t. I have no comment on mt about these, since I didn’t keep my account long enough to get a real sample-size for that sort of info.

Okay, so we’ve got a clustered web-stack for $20/month, that’s coo,l right? I still think it sounds like a good idea, but what do you get for that? Oh, a 64 meg Rails container to run mongrel in? Well that isn’t going to go very far; better bump that up, so now we are looking at $45/month and we are getting into low-end full VPS solutions.

That’s the problem I had basically – the lack of flexibility of the GridServer environment didn’t make the bargain seem worth it. For instance, there is no set-up for clustering Mongrels, and that shoots down the easily scalable claim. There is no way to run Trac, since it’s python, they don’t have mod_python installed, and the clustered environment doesn’t work with fcgi, by it’s very nature. It looked like Collaboa, a Rails alternative wouldn’t work, either, since it requires the ruby svn bindings, and you can’t install compiled gems onto the Grid.

Overall, I’d say once they get some of the initial weak points worked out it will probably be worth looking at again. Until then pick your poison for affordable VPS. You guys can Google™ that as well as I can, and will probably get the same list of Rails friendly hosting setups that come up every time that question is raised. In fact, if someone is happy with their affordable VPS, for running Rails, please comment here, as I am still stuck on Dreamhost.