Using REGEX to parse a search string with boolean operators

You can use a simple regular expressions to yank out tokens, with the quotes and everything in them, and then pretty them up before you use them. Something like this:

function query_tokens($query)
{
    $regex = '/-?"[\pL\s]+"|-?\pL+/';

    preg_match_all($regex, $query, $tokens, PREG_SET_ORDER);

    foreach ($tokens as & $token)
    {
        $token = array_shift($token);

        $modifier = NULL;

        if ($token[0] === '-' || $token[0] === '+')
        {
            $modifier = $token[0];

            $token = substr($token, 1);
        }
        if ($token[0] === '"')
        {
            $token = trim($token, '"');
        }
        $token = $modifier.$token;
    }

    return $tokens;
}

The string used and result from the function:

var_dump(query_tokens('"this is some" text here is -more -"exlude me"'));
array (size=6)
  0 => string 'this is some' (length=12)
  1 => string 'text' (length=4)
  2 => string 'here' (length=4)
  3 => string 'is' (length=2)
  4 => string '-more' (length=5)
  5 => string '-exlude me' (length=10)

Regular expressions are great, but sometimes they can make things more complicated than they need to be.

You cannot capture '-exclude me' using a regular expression because matches are always consecutive. At best, you could modify the regular expression to match the '-more' token as such:

(?<!")-?\b\w+\b|(?<=")\b[^"]+