javastringstringtokenizer

StringTokenizer only storing first token?


I have tried to split a string into tokens using StringTokenizer but when I try to count or print the tokens, it only includes the first word. Any help would be appreciated.

      Scanner s = new Scanner(System.in);
      System.out.println("ENTER STRING:");
      text = s.next();
      StringTokenizer str = new StringTokenizer(text," ");
      System.out.println(str.countTokens());
      while(str.hasMoreTokens())
       { System.out.println(str.nextToken()); }

Output:

ENTER STRING: Hi welcome to java

1

Hi


Solution

  • Your problem is not with the StringTokinizer. It’s here:

        text = s.next();
    

    You scanner also delimits at whitespace. So the above line only reads the first word, Hi. This is why the tokenizer only includes this word.

    The solution is to use the nextLine method of the scanner. It reads the full input line.

    StringTokenizer is legacy

    The StringTokenizer is legacy and only included with Java for backward compatibility. It‘s recommended to use String.split() for separating a string into tokens.

    Full example:

        Scanner s = new Scanner(System.in);
        System.out.println("Enter string:");
        String text = s.nextLine();
        String[] tokens = text.split(" ");
        System.out.println(tokens.length);
        for (String t : tokens) {
            System.out.println(t);
        }
    

    Example session:

    Enter string:
    Hi welcome to java
    4
    Hi
    welcome
    to
    java