The Gemcutter Update

February 2010 Changelog

2010-03-05T00:00:00-08:00

Welcome to the second monthly changelog post! February was a big month for Gemcutter: a main goal of the project was realized with the release of RubyGems 1.3.6 along with plenty of other work. Another big change is that gem yank has been implemented finally with gemcutter 0.5.0, and John Trupiano made a great screencast to show off how it works:

Stats

Gem Downloads: 4,396,743 (up 299,027)
Gems Pushed: 4,308
S3 Bandwidth Served: 567.103 GB
CloudFront US Node Served: 634.189 GB
CloudFront Europe Node Served: 327.385 GB
CloudFront Japan Node Served: 64.744 GB
CloudFront Hong Kong Node Served: 47.008 GB
Commits: 40

Features

gem yank: Plenty of work was done based on discussion on the Gemcutter mailing list on finally being able to remove gems from the site’s index. The motivation behind why it works this way is best explained by On Retagging from the git tag manpage. Here’s a small summary of the changes:

If you’ve botched a gem release, you can use gem yank [gemname] to remove it from the index. Fix the problem, bump by a patch version, and push the newly built gem. You’re not going to run out of version numbers any time soon!
The gem is still available for download, and the gem is not deleted from S3. This prevents a _why type situation where important community gems could suddenly disappear, and allows other gem authors to be able to use it if necessary.
If you need a gem perma-deleted, open a support request at help.rubygems.org and we’ll take care of it.
Yanked a gem you didn’t mean to? Run gem yank --undo to put the gem back into the index.

RubyGems Integration: RubyGems.org is now the default gem host in RubyGems, gem push and gem owner have been merged into RubyGems proper. Plenty of copy and documentation changes were done too.

Fixes

Atomic Versions: Re-pushing of gems is now disabled since gem yank is implemented.

Staging Server: We finally have a staging server to test out major changes. We upgraded the site to the latest Passenger and REE too on production!

Load Path Fixes: Some cleanup of the LOAD_PATH has been performed for the latest gemcutter gem.

Gemcutter Gem Refactoring: As of RubyGems 1.3.6, utilities have been added to help use the Gemcutter API, so gem webhook and gem yank now take advantage of this.

Destroy Subscriptions on RubyGem Destroy: This was causing some problems on user dashboards.

Force Redirects for Other Domains: Visiting pages on the web frontend at gemcutter.org or gems.rubyforge.org redirect you to rubygems.org.

Updating Legacy Indexes: A new RubyGems release means that the old legacy indexes had to be updated for old RG clients to be able to auto-update.

RubyGems.org move complete

2010-02-20T00:00:00-08:00

As you may have noticed, Gemcutter has now become RubyGems.org. This transition has been in the works for several months now, and the new RubyGems 1.3.6 release brings it to a close. The original goal of Gemcutter was to improve the RubyGem hosting scene for everyone in the Ruby community, and it’s been realized now.

For those concerned with what’s going to change, here’s some basic information about what’s happened to bring you up to speed:

gemcutter.org, gems.rubyforge.org, and rubygems.org all point to the same place
gem serving/installing will continue to work for all 3 domains
rubygems.org is the default gem source as of RubyGems 1.3.6
rubygems.org is the main web frontend now, and the other two sites redirect to it
gem owner and gem push have been merged into RubyGems proper.
The gemcutter gem as of 0.4.0 just has gem webhook (and soon, gem yank!)
The GitHub project, Twitter account, etc name will remain the same

Going forward, we’ll eventually be using gemcutter.org for gem forking/subdomains, and gems.rubyforge.org will probably remain inactive. I’d also like to say thanks to Eric Hodel, Tom Copeland, and John Barnette, their help with making this all happen has been immeasurable.

If you’ve got questions about the transition, let us know in the comments!

January 2010 Changelog

2010-02-01T00:00:00-08:00

Hey folks, this is going to be a new monthly roundup of what went on at your community gem host. Thanks for the hard work from all of our contributors, and keep it up! If you’re interested in helping out check out our contribution guidelines.

Stats

Gem Downloads: 4,097,716
Gems Pushed: 4,361
Commits: 67
Files Changed: 206
Lines Added: 1,471
Lines Removed: 4,528
New Contributors: 5

Features

Webhooks: Get notifications for specific gems or all gems when new versions are pushed. Read up here.

CloudFront: We’re now serving all gems older than a day from CloudFront, which should speed up downloads immensely.

RubyGems Support Site: Need help with Gemcutter or RubyGems? Look no further

Metrics/Caliper Integration: Shiny new button on every gem’s page, and Caliper generates metric_fu results for every pushed gem. Awesome.

Gem Payload Additions: Gem URI, Project URI, Version Downloads, Dependencies: Plenty added to the API hash, check out an example

Search API method: An alternative to gem search that hooks into the same search box on the site at /api/v1/search.(xml|json)?query="SOME GEM"

XML API Payload: For those who enjoy 3-letter acronyms when querying their gems. More on the API docs.

Fixes

Switched to postfix: We used to just use GMail for our mail server when we were on Heroku, but now we can use the server’s postfix server to send mail.

Strip string on search: Putting spaces in search queries would give bad results, so now they’re stripped away.

More legacy indexes: Some of the legacy indexes that old versions of RubyGems were not being served up. They’re now all served up with just rubygems-update in them.

Redirecting to all indexes on S3: Instead of trying to cache the indexes through the Hostess, we now redirect to S3 for all of the indexes we can. See for yourself with gem install -V.

Moving gems to /downloads: Search engine crawlers were bumping download counts, so we’ve moved gem downloads on rubygems#show to /downloads and added that to robots.txt so they don’t bump up the counts.

Various CSS fixes on rubygems#show: Too many authors, downloads, or even a long title would break the page layout, hopefully that’s fixed now.

ILIKE: This was put in since Gemcutter uses PostgreSQL but would hinder contributors wanting to use MySQL. It’s now removed and we should be DB agnostic again.

RubyForge Account Migration

2009-12-10T00:00:00-08:00

Thanks to some hard work from David Black and Tom Copeland, all of the RubyForge accounts have now been imported into Gemcutter’s database. This move ushers in a few changes:

1) You can log in with your RubyForge account info if you haven’t made a Gemcutter account yet. If you had both a Gemcutter and RubyForge account under the same email, your account has not been changed.
2) All of the project ownerships from RubyForge have been transferred over to Gemcutter. If you’re still having trouble gaining access to your RubyForge gems/projects, feel free to shoot nick@gemcutter.org an email.
3) gem migrate is now deprecated in the Gemcutter gem as of version 0.2.1.
4) We now have usernames for the site, which will allow us to do all sorts of neat features! (dashboard, subdomains, etc)

As always, if you’re looking to help out with the project feel free to fork the repo or hop in #gemcutter on irc.freenode.net to see how you can help out.

Gemcutter Security Alert - gem update gemcutter

2009-12-08T00:00:00-08:00

Early last month, thanks to a hat-tip from Tim Carey-Smith, a vulnerability was found in Gemcutter’s ownership system that allowed access to other users’ API keys. David Dollar was quick to fix the issue, but due to miscommunication on my part, the Gemcutter API keys were never reset in response to the security hole.

I’ve now reset all Gemcutter API keys for all users. In order to push your gems once again, you’ll need to simply update the Gemcutter gem:

gem update gemcutter

When you need to gem push again, you’ll be prompted to sign in and your new API key will be fetched. That’s all you should need to do in order to be back up to speed with releasing gems. No known compromises of gems has happened to our knowledge during the time since the vulnerability was closed. Downloading gems has not been affected by this issue.

Going forward, I’ve set up a new address for reporting security issues with the site at security@gemcutter.org. Please report any code vulnerabilities you find there for now on. A security policy akin to Rails’ regarding vulnerability issues will be set up soon and announced on the mailing list once completed. Sorry for the inconvenience here folks, and thanks for using Gemcutter.

Moving Forward

2009-11-16T00:00:00-08:00

We’re getting ready for the RubyGems.org transition, and the first thing that needs to happen is a DNS switch for Gemcutter from Heroku to our new home at RackSpace. If you’re just downloading gems from Gemcutter, your gem installs will continue to work thanks to our maintenance mode setup. If you’re using the Gemcutter gem, you will need to upgrade it to continue to push. Run gem update gemcutter and you should be all set. Going forward, the gem plugins will be merged into RubyGems in order lower the barrier further for gem publication.

The entire Heroku team has been excited to see the site blossom on the platform, and as huge supporters of the Ruby community they want nothing but this effort to succeed. To further the integration of Gemcutter with RubyForge, one of the major decisions made was that Gemcutter’s database would become the central database for RubyForge. We’ll be integrating projects, mailing lists, and other useful features. To facilitate this, we needed to move Gemcutter and RubyForge into the same datacenter, and decided it was easier to move Gemcutter than to move all of RubyForge to Heroku. I’d just like to say thanks to the Heroku team for the help and support so far, our decision to move off Heroku has nothing to do with the quality or capabilities of the Heroku platform.

Since there was no lock-in with the Heroku platform, no modifications were necessary in order to move. Several of the optimizations the Heroku platform encouraged have actually helped make this transition really easy:

1) Using S3 and the read-only file system helped make the migration onto a new platform very easy, since only the app code needed to move over.
2) Delayed::Job for background processing meant that we could simply run the workers on the new server and `gem push` would still work.
3) The simplicity of heroku db:pull means that migrating Gemcutter’s database will go extremely quickly.

Heroku’s been a huge part of Gemcutter and will continue to be as the project moves forward. As more of the RubyForge platform is migrated over to Ruby, hopefully there will be a chance to return to the ease of deployment and management that Heroku offers.

Transition

2009-10-26T00:00:00-07:00

It’s time to talk about the future of gem hosting. I started Gemcutter as an effort to improve RubyGem hosting for everyone. I wasn’t happy with the ways that RubyForge or GitHub was currently handling it, and I felt there could be a better way of fixing this problem. I set out to make gem indexing instantaneous, information about gems more accessible, and to provide an open API for gem publishing. The overall goal was to eventually take over gem hosting on RubyForge, and work with the established community instead of trying to fight against them.

On September 25, I revealed that Gemcutter would be moving to http://rubygems.org, and becoming RubyGems.org: your community gem host. Today, I’d like to announce that Ruby Central has agreed to support RubyGems.org in becoming the default gem host for the community. We’re still in the process of working out all of the details of this migration, and it will involve several moves:

http://rubygems.org will replace http://gems.rubyforge.org as the default gem host in RubyGems.
Gem publishing off RubyForge will continue to work for the time being.
We’ll be merging user accounts from RubyForge, so you’ll be able to log into RubyGems.org with your RubyForge login credentials. Your gem ownerships will also be transferred over.

So, what does this mean for RubyForge? The Ruby-specific functionality and data will be moved into RubyGems.org, and the parts that other hosting sites (GitHub, Google Code, SourceForge) can do better will be pruned away. Migration paths for those projects will be provided, we’re not throwing any switches without warning. RubyGems.org will not be gaining any “bloat” from rewritten RubyForge features.

The goal right now is to have the majority of this transition done before RubyConf (November 19). More information about dates and migration information, especially regarding when RubyForge features will be put into read-only mode will be placed here on the wiki.

I’d like to thank everyone involved for their support. I’m proud to say that RubyGems.org will remain at the center of the Ruby community going forward, and I invite you to join in with your own contributions. This is our gem host now, let’s make it awesome.

Kinetic Energy

2009-09-25T00:00:00-07:00

Welcome to the first edition of The Update! This blog is going to cover the latest features, news, and awesome gems coming out of Gemcutter. Let’s get started.

Announcements

As you can tell, the redesign is live! Thanks to Thoughtbot for donating design time to the project and making sure it looks great.

The next big piece of news is that this site will be moving to http://rubygems.org soon. (The current site will stay live at http://docs.rubygems.org). We’re currently working with the RubyGems team on this, and it will require a little bit of redirecting and rebranding on our part. Stay tuned here for updates!

For those wondering about replacing/redirecting gems.rubyforge.org, there’s a discussion going on at rubygems-developers about this very subject. If you’re a fan of Gemcutter, check out this proposal and lend your support!

We’re also going to use this space to feature gems that are new (and of course, some that are old) that are hosted on the site. This will be a column in the style of GitHub Rebase, so if you’ve got ridiculous Ruby images or great gems, let me know at nick@gemcutter.org.

Features

Much has happened since our launch. (No, that wasn’t planned.) Since then we’ve had a lot of great new features added in:

Adding/removing owners on a gem
Subscribe to gems to get a feed of what’s been updated
Enhanced search with descriptions
Prerelease gem support
Better gem documentation
Faster gem indexing for newly pushed gems
Improved gem serving thanks to rack-cache and Varnish
HTTP Proxy support for the gem

And so much more that can’t fit here.

Upcoming

Here’s a sneak peek at some features slated for the future. No guarantees here on what could be finished, but these are some of the ideas in the works. The great part is that you can contribute and make these (or your own) a reality!

Deleting gems
Full text search of READMEs from gems
JRuby support
Gravatar Integration (everyone loves avatars!)
Stats page
Better download tracking for individual gem versions

If you’ve got your own ideas, feel free to submit an issue!

Thanks!

Enough thanks can’t be said for the great start that this project has had so far. This will have to suffice: Thanks to you for using it, our over 15 contributors for their hard work, and Heroku for supporting the project.

If you haven’t yet, gem install gemcutter and gem tumble away! See you next time.