sphinx

Sphinx (search engine) 3 not reading CSV files


I'm trying to get Sphinx to index a CSV file. At this point, I'm trying to get the most trivial example working, but having no luck.

I'm using: Sphinx 3.3.1 (commit b72d67b)

This is my conf file

source csvsrc
{
    type = csvpipe
    csvpipe_command = cat /home/ec2-user/sphinx/etc/example.csv
    csvpipe_header = 1
    csvpipe_attr_uint = gid
    csvpipe_attr_string = title
    csvpipe_attr_string = content
}

index csvtest
{
    type            = rt
    source          = csvsrc
    path            = /var/data/test5
    morphology      = stem_en

    rt_field = gid
    rt_field = title
    rt_field = content

}

indexer
{
    mem_limit       = 128M
}

searchd
{
    listen          = 9312
    listen          = 9306:mysql41
    log             = /var/log/searchd.log
    query_log       = /var/log/query.log
    pid_file        = /var/log/searchd.pid
    binlog_path     = /var/data
}

This is the csv file I'm trying to read

gid, title, content
11, hello world, document number one
124, hello again, document number two
125, hello now, This is some stuff
126, hello cow, more test stuff and things
127, hello suess, box and sox and goats and moats
128, hello raven, nevermore said the thing

I'm running the command ../bin/searchd --config /home/ec2-user/sphinx/etc/sphinx.conf Which responds with:

using config file '/home/ec2-user/sphinx/etc/sphinx.conf'...
listening on all interfaces, port=9312
listening on all interfaces, port=9306
precaching index 'csvtest'
precached 1 indexes using 1 threads in 0.0 sec

To launch the engine, if I run the indexer with the following:

../bin/indexer --config /home/ec2-user/sphinx/etc/sphinx.conf

It responds as below

Sphinx 3.3.1 (commit b72d67b)
Copyright (c) 2001-2020, Andrew Aksyonoff
Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

ERROR: nothing to do.

All of that is as expected, but when I connect with a MySQL client, I find the index well-formed, but empty

$ mysql -h127.0.0.1 -P9306
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 3.3.1 (commit b72d67b) 

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [(none)]> describe csvtest
    -> ;
+---------+--------+------------+------+
| Field   | Type   | Properties | Key  |
+---------+--------+------------+------+
| id      | bigint |            |      |
| gid     | field  | indexed    |      |
| title   | field  | indexed    |      |
| content | field  | indexed    |      |
+---------+--------+------------+------+
4 rows in set (0.00 sec)



MySQL [(none)]> select * from csvtest;
Empty set (0.00 sec)

What do I need to change to actually get the csv data to load into the index?


Solution

  • @barryhunter solved it in the comments.

    I needed --all on the indexer to make it actually index.