Stop Logstash process automatically after imported all data

Question

Situation:

• I'm importing data to Elasticsearch via Logstash at 12 pm manually every day.
• I understand there is no "close" on Logstash because ideally, you would want to continuously send data to the server.
• I am using elk-docker as my ELK stack.
• I wrote a shell script that sends a command to a docker container to execute the following:

dailyImport.sh

docker exec -it $DOCKER_CONTAINER_NAME opt/logstash/bin/logstash --path.data /tmp/logstash/data -e \
'input {
    file {
        path => "'$OUTPUT_PATH'"
        start_position => "beginning"
        sincedb_path => "/dev/null"
        mode => "read"
        file_completed_action => "delete"
    }
}
filter {
    csv {
        separator => ","
        columns => ["foo", "bar", "foo2", "bar2"]
    }

}

output {
    elasticsearch{
        hosts => "localhost:9200"
        index => "foo"
        document_type => "foo"
    }
    stdout {}
}'

What I have tried and understood:

• I have read that adding read mode and file_completed_action to delete would stop the operation, I tried it but it didn't work.
• I would still need to send Ctrl + C manually to stop the pipeline. e.g:

^C[2019-02-21T15:49:07,787][WARN ][logstash.runner          ] SIGINT received. Shutting down.
[2019-02-21T15:49:07,899][INFO ][filewatch.observingread  ] QUIT - closing all files and shutting down.
[2019-02-21T15:49:09,764][INFO ][logstash.pipeline        ] Pipeline has terminated {:pipeline_id=>"main", :thread=>"#<Thread:0x6bdb92ea run>"}
Done

I have read that I could do the following, but don't know how:

• Monitor the sincedb file to check when Logstash has reached EOF, then kill Logstash.
• Use the stdin input instead. Logstash will shut down by itself when stdin has been closed and all inputs has been processed. On the flip side, it Logstash dies for whatever reason you don't know how much it has processed.

Reference: https://discuss.elastic.co/t/stop-logstash-after-processing-the-file/84959

What I want:

• I don't need a fancy progress bar to tell me how much data I have imported (against the input file).
• I only want to end the operation when "it's done" and maybe send a Ctrl + C when it reaches the EOF or "finished importing".

Answer 1

for file input in read mode there's recently a way to exit the process upon reading all files, just set:

input { file { exit_after_read => true } }

https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-exit_after_read