Thumbs_up gem rails - how to take back a vote?

I finally got my votes being submitted correctly via ajax, however I can' seem to unvote at all. Here's my models:

class User < ActiveRecord::Base

    acts_as_voter
end

class Vendor < ActiveRecord::Base

    acts_as_voteable
end

My routes:

resources :vendors do
    collection do
      post :vote_for_vendor
      delete :vote_against_vendor
    end
  end

My controller:

class VendorsController < ApplicationController

    def vote_for_vendor
        begin
            vendor = Vendor.find(params[:vendor_id])
            current_user.vote_for(vendor)
            render :nothing => true, :status => 200
        rescue ActiveRecord::RecordInvalid
            render :nothing => true, :status => 404
        end
    end

    def vote_against_vendor
        begin
            vendor = Vendor.find(params[:vendor_id])
            current_user.vote_against(vendor)
            render :nothing => true, :status => 404
        rescue ActiveRecord::RecordInvalid
            render :nothing => true, :status => 404
        end
    end
end

My view:

<table class="table table-condensed table-hover">
    <tr>
        <th>Name</th>
        <th>Address</th>
        <th>Favorite?</th>
    </tr>
        <% @vendors.each do |v| %>
    <tr>
        <td><%= v.name %></td>
        <td><%= v.address %></td>
        <td id="toggle">
            <% if current_user.voted_for?(v) %>
                <%= button_to "Unlike", { :controller => :vendors, :action => 'vote_against_vendor', :vendor_id => v.id},
                                        { :method => 'delete', :remote => true }%>
            <% else %>
                <%= button_to "Like", { :controller => :vendors, :action => 'vote_for_vendor', :vendor_id => v.id},
                                    { :method => 'create', :remote => true} %>
            <% end %>
        </td>
    </tr>
        <% end %>
</table>

So how can I unvote for a vendor? I read in the thumbs_up docs that by default a voter can only vote once. Does this mean that a user who votes_for can't then vote_against? The purpose of my implementation is to have a favorite or like button.

Here's what I'm getting from the server upon clicking 'Unlike':

Started DELETE "/vendors/vote_against_vendor?vendor_id=1" for 127.0.0.1 at 2013-09-06 14:34:02 -0700
Processing by VendorsController#vote_against_vendor as JS
  Parameters: {"authenticity_token"=>"/ZdePkPe+4rM8thUa+81hEL68cw1CJn93P0LQqEMC3s=", "vendor_id"=>"1"}
  Vendor Load (0.2ms)  SELECT "vendors".* FROM "vendors" WHERE "vendors"."id" = ? LIMIT 1  [["id", "1"]]
  User Load (0.3ms)  SELECT "users".* FROM "users" WHERE "users"."id" = 1 LIMIT 1
   (0.1ms)  begin transaction
  Vote Exists (0.2ms)  SELECT 1 AS one FROM "votes" WHERE ("votes"."voteable_id" = 1 AND "votes"."voteable_type" = 'Vendor' AND "votes"."voter_type" = 'User' AND "votes"."voter_id" = 1) LIMIT 1
   (0.1ms)  rollback transaction
  Rendered text template (0.0ms)
Completed 404 Not Found in 7ms (Views: 0.6ms | ActiveRecord: 0.7ms | Solr: 0.0ms)

Pro Tip: Read through the documentation very thoroughly before you look silly and ask a dumb quesiton. Just use the unvote_for method to get this done. Not sure how I missed it the first time.

In vote_against_vendor, directly after you call vote_against, you do this:

render :nothing => true, :status => 404

When you should probably do this:

render :nothing => true, :status => 200