Menu
I want to be able to tokenize an input string from a text box to do my query. Example: user enters "abc xyz 123" in the text box. I want to do this:
SELECT * FROM database WHERE Name contains "abc" AND "xyz" AND "123"
-- as opposed to containing "abc xyz 123"
-- please ignore my sql syntax, I am an absolute beginner
Thanks.
298
use SQL::Tokenizer qw(tokenize_sql) ; my @tokens = tokenize_sql( $query ); my $tokens = tokenize_sql( $query ); $tokens = tokenize_sql( $query , $remove_white_tokens );
Tokenization is the act of breaking up a sequence of strings into pieces such as words, keywords, phrases, symbols and other elements called tokens. Tokens can be individual words, phrases or even whole sentences. In the process of tokenization, some characters like punctuation marks are discarded.
The common solution to tokenize a string in C++ is using std::istringstream , which is a stream class to operate on strings. The following code extract tokens from the stream using the extraction operator and insert them into a container.
To perform sentence tokenization, we can use the re. split() function. This will split the text into sentences by passing a pattern into it.
Using a string split function (for example, like this one), you could have something like this:
SELECT t.*
FROM atable t
INNER JOIN dbo.Split(@UserInput, ' ') s ON t.Name LIKE '%' + s.Data + '%'
One possible solution is that you take the string and split it into your tokens and the for each token insert it into a temp table and then join you temp table to your search table and in the join do a like '%' + tokenColumn + '%' to get all your rows that contain a value from your tokens
Here's an example of splitting the string: http://blogs.microsoft.co.il/blogs/itai/archive/2009/02/01/t-sql-split-function.aspx
37
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
