How to hide Fivestar rating field programatically after first comment?

I'd like to hide the Fivestar rating field (called 'field_stars') in my comment form when a user comments on a node on which he/she's already commented. I have a flag called Commented on my site and a rule that flags a node on behalf of the commentator when someone commented on it (which works perfectly). I tried to fix my problem, based on the information I found here:

function hiderating_form_alter(&$form, &$form_state, $form_id) {
  global $user;
  $flag = flag_get_flag('commented') or die('no "commented" flag defined');
  if ($form_id == "comment_node_stuff_form") {
   if ($flag->is_flagged($form['#node']->uid, $user->uid)) { 
      unset($form['field_stars']);
    }
  }
}

Unfortunately, it doesn't change anything (I don't get any errors either). What did I do wrong?

After debugging with @jerdiggity's code I get:

DEBUG: form id "comment_node_stuff_form" found. Debugging...
DEBUG: The code $flag->is_flagged($form["#node"]->uid, $user->uid) is returning FALSE.

(of course I get the DEBUG: final form structure: too, but that's way to long to paste here). I don't understand why the flagging isn't recognized, as it is stored in the database...

Try adding these debug values to get to the root of the problem (not on a live server, of course... Only in a development environment):

function hiderating_form_alter(&$form, &$form_state, $form_id) {
  global $user;
  if ($form_id == "comment_node_stuff_form") {
    // I'm not sure why the examples make use of "die" because that will stop the entire
    // site from continuing any further.
    $flag = flag_get_flag('commented');

    // ** Begin debug ** //
    // Notify us that we've at least got the right form.
    drupal_set_message(t('DEBUG: form id "comment_node_stuff_form" found. Debugging...'), 'warning');
    // Warn us if $flag is giving us issues
    if (!$flag) {
      drupal_set_message(t('DEBUG: No "commented" flag defined.'), 'warning');
    }
    // Check form components
    if (!isset($form['#node']) || !$form['#node']) {
      drupal_set_message(t('DEBUG: Something is wrong with or missing from $form["#node"]'), 'warning'); 
    }
    // Check more form components
    if (!isset($form['#node']->uid)) {
      drupal_set_message(t('DEBUG: $form["#node"]->uid is not set.'), 'warning'); 
    }
    // Again
    if (!isset($form['field_stars']) || empty($form['field_stars'])) {
      drupal_set_message(t('DEBUG: <em>$form["field_stars"]</em> is not set or is empty/false.'), 'warning'); 
    }
    // Check flag stuff
    if (!$flag->is_flagged($form['#node']->nid, $user->uid)) {
      drupal_set_message(t('DEBUG: The code <em>$flag->is_flagged($form["#node"]->nid, $user->uid)</em> is returning FALSE.'), 'warning'); 
    }
    // Again...
    if (!$flag->is_flagged($form['#node']->uid, $user->uid)) {
      drupal_set_message(t('DEBUG: The code <em>$flag->is_flagged($form["#node"]->uid, $user->uid)</em> is returning FALSE.'), 'warning'); 
    }
    // ** End debug ** //

    // Changing this to nid instead of uid
    if ($flag->is_flagged($form['#node']->nid, $user->uid)) { 
      unset($form['field_stars']);
    }
    // One more debug item:
    drupal_set_message('DEBUG: final form structure:<br/><pre>' . check_plain(print_r($form, 1)) . '</pre>', 'warning');
  }
}