SPL Parser Package Modules

This page lists documentation for the modules contained in spl_parser package.

spl_parser.cli

Module cli defines functionality for the CLI part of spl_parser. The CLI interface defines two operating modes (generate/view). Both operating modes can be used with a remote Splunk server or with local files.

validate_file(ctx, params, file)

Callback to validate provided local file.

Parameters

  • ctx – context object obtained from click

  • params – parameter object obtained from click

  • file (str) – name of the local file to validate

Raises

click.BadParameter – if the provided file is of json, neither conf format

Returns

the provided filename, if valid

Return type

str

validate_url(ctx, paarams, url)

Callback to validate provided URL.

Performs validation of the URL towards a regular expresssion.

Parameters

  • ctx – context object obtained from click

  • params – parameter object obtained from click

  • url (str) – URL to validate

Raises

click.BadParameter – if the provided URL has wrong format

Returns

the provided URL, if valid

Return type

str

spl_parser.cli_print

This module defines functions for easy and pretty printing of messages to terminal. This is ensured using click package.

enable_debug()

Enable debugging.

log_message(level, message)

Print log message to the terminal by using click.

Debugging must be enabled to print DEBUG messages.

Parameters

  • level (str) – The level of the message. One of ERROR, WARNING, INFO, or DEBUG

  • message (str) – The message to print.

print_kv(key, value)

Print key-value pair to the terminal.

Key is printed in bold style, whereas the value is printed normally.

Parameters

  • key (str) –

  • value (str) –

spl_parser.exceptions

Defines specific exceptions used in spl_parser package.

exception AuthenticationError

Used when there is error in authentication towards a remote Splunk server.

exception CommandNotFoundError(message)

Error signalizing that command could not be retrieved.

Message should specify name of the command.

exception ConnectionError

Generic error which occurs in communication with a remote Splunk server.

exception InitError

Error occuring during initialization phase.

Such error can be related to the Pseudo-BNF grammar used for parsing, or a tmLanguage template.

exception ParsingError(message)

Error that occurs during parsing.

Message can specify a name of the command or file where the parsing error occurred.

exception SplParserError

Generic exception used with spl_parser package.

spl_parser.searchbnf_parser

This module defines functions that allow to parse searchbnf definitions in various fomats (JSON/conf) into their Python representations for further operations.

parse_conf(conf_file, parts=[])

Parse searchbnf.conf file.

The provided searchbnf.conf file is processed and parsed to return all contained SPL terms. Optionally, only a subset of the file may be processed, by specifying a list containing names of the sections to be processed.

Parameters

  • conf_file (str) – path to the searchbnf.conf file

  • parts (list, optional) – names of parts to be processed. Defaults to empty list(), meaning the whole file.

Returns

dictionary of SPLTerms objects

Return type

dict

parse_json(json_data, parts=[])

Parse searchbnf data in json format.

The provided searchbnf data in json is processed and parsed to return all contained SPL terms. Optionally, only a subset of the file may be processed, by specifying a list containing names of the terms to be processed.

Parameters

  • json_data (dict) – json data containing searchbnf definitions

  • parts (list, optional) – names of parts to be processed. Defaults to empty list(), meaning all the data.

Returns

dictionary of SPLTerms objects

Return type

dict

parse_spl_term(name, data)

Parse an SPL term from the pre-loaded data.

The provided data must have a dict-like format, representing a single SPL term in a standard fomat found in searchbnf.conf or searchbnf.json files. If the provided term is a command, an SPLCommand object is created instead of SPLTerm.

Parameters

  • name (str) – name of the term to parse

  • data (dict) – dict-like data representing a single SPL term

Returns

an initialized SPLTerm object

Return type

SPLTerm

spl_parser.spl_objects

Module spl_objects defines SPLTerm and SPLCommand classes that serve as Python representations for SPL commands and terms.

class SPLCommand(name, syntax)

Object representing an SPL command.

Parameters

  • name (str) – name of the SPL command

  • syntax (str) – syntax for the SPL command

parse(syntax_trees)

Parse full properties of the SPL command from syntax trees.

Details for the SPL command (its arguments, functions, operators) are parsed from syntax trees. The trees related to the processed SPL command are chosen from all available trees. If the command cannot be parsed correctly, a warning is printed to the terminal.

Parameters

syntax_trees (dict) – dictionary containing pre-built syntax trees

print()

Pretty print the SPL command details to terminal.

class SPLTerm(name, syntax)

Object representing a generic SPL term.

Parameters

  • name (str) – name of the term

  • syntax (str) – syntax for the term

get_bnf_syntax()

Retrieve Pseudo-BNF syntax from the term representation.

Returns

syntax string in Pseudo-BNF format

Return type

str

print()

Pretty print the SPL term details to terminal.

spl_parser.spl_resource

Module spl_resource contains definition of the workflow for the main functionalities of the package - generating the grammar and viewing commands. These functionalities are performed on objects called SplResource. Based on the operation mode, a LocalSplResource, or a RemoteSplResource is used.

class LocalSplResource(file)

Class defining an SPL resource based on a local file.

fetch_spl_terms(spl_terms=[])

Fetch information for SPL terms from the resource.

Parameters

spl_terms (list, optional) – names of SPL terms to be fetched. Defaults to empty list(), meaning all available resources.

class RemoteSplResource(url, username, password)

Class defining an SPL resource based on a remote Splunk server.

Parameters

  • url (str) – URL of a remote Splunk server

  • username (str) – username to authenticate to the remote Splunk server

  • password (str) – password corresponding to the username

async async_fetch_spl_term(spl_term)

Asynchronously fetch details about an SPL term.

Parameters

spl_term (SPLTerm) – SPL term to retrieve

async async_fetch_spl_terms(spl_terms)

Asynchronously fetch details about SPL terms.

Parameters

spl_terms (list) – SPL terms to retrieve

async async_get(url)

Perform asynchronous GET request.

Parameters

url (str) – URL to use

Returns

JSON data received as response

Return type

dict

build_url(term_name=None)

Build a URL for the searchbnf file on the remote Splunk server.

If a term is specified, builds URL for the specific term, otherwise builds URL for all terms.

Parameters

term_name (str, optional) – name of the term. Defaults to None (all terms).

Returns

the build URL

Return type

str

fetch_spl_terms(spl_terms=[])

Fetch information for SPL terms from the resource.

Parameters

spl_terms (list, optional) – names of SPL terms to be fetched. Defaults to empty list(), meaning all available resources.

class SplResource

Base class defining SplResource

fetch_spl_terms(spl_terms=[])

Fetch information for SPL terms from the resource.

Parameters

spl_terms (list, optional) – names of SPL terms to be fetched. Defaults to empty list(), meaning all available resources.

generate_grammar(outfile)

Generate a tmLanguage grammar for SPL.

Performs all necessary steps to fetch, process, build and generate a tmLanguage grammar file for SPL. Informational messages are printed to terminal during the generation process.

Parameters

outfile (str) – name of the file to save the generated grammar into

view_command(command)

View details about an SPL command.

Performs all necessary steps to fetch, process and view information about the provided command. The command details and informational messages are printed to terminal.

Parameters

command (str) – name of the command to view

Raises

CommandNotFoundError – if the specified command was not found

spl_parser.syntax_parser

This module defines the core functions responsible for the parsing of Pseudo-BnF grammar, building syntax trees from it and manipulating with them.

class LarkParser(grammar)

Class serving as a simple wrapper for Lark.

Parameters

grammar (str) – loaded contents of the file defining Lark grammar for Pseudo-BNF.

parse(expression)

Parse a Pseudo-BNF expression using Lark.

Parameters

expression (str) – a Pseudo-BNF expression to parse

Returns

created syntax tree

Return type

lark.Tree

build_syntax_trees(pseudo_bnf, spl_terms)

Build syntax trees from Pseudo-BNF definitions.

Provided SPLTerm objects should contain Pseudo-PNF definition of the syntax. This function builds syntax trees from the syntax definitions, based on the provided Lark grammar.

Parameters

  • pseudo_bnf (str) – loaded contents of the file defining Lark grammar for Pseudo-BNF.

  • spl_terms (dict) – dictionary containing initialized SPLTerm objects

Raises

InitError – if an error with the Lark grammar occurs

Returns

dictionary containing lark.Tree objects, representing syntax of the SPL terms

Return type

dict

choose_trees(names, trees)

Choose syntax trees based on their names.

Parameters

  • names (list) – names of the trees to choose.

  • trees (dict) – group of available syntax trees

Returns

a copy of tree dict containing only trees, whose names were present in the list

Return type

dict

Find all related syntax trees to that specified by name.

Recursively search all available trees to resolve the most accurate representation of a SPL term specified by name.

Parameters

  • name (str) – name of a SPL term to process

  • trees (dict) – available syntax trees to search

Returns

all found related trees

Return type

dict

get_token_data(subtree)

Get data from the leaf of a subtree.

Parameters

subtree (lark.Tree) – [description]

Returns

String representation of the data contained in the first leaf of the subtree.

Return type

str

parse_syntax_tree(tree)

Parse a single syntax tree.

Looks for possible arguments, operators and functions contained in the tree representation of an SPL command.

Parameters

tree (lark.Tree) – syntax tree representing SPL command

Returns

tuple of found arguments, functions and opreators

Return type

tuple

spl_parser.tmlanguage_generator

Module defining functionality related to generating a TmLanguage grammar for SPL.

class TmLanguageGenerator(template_data)

Class defining a TmLanguage generator.

Parameters

template_data (str) – contents of the file defining a tmLanguage template

add_command(spl_command)

Add SPLCommand to the grammar.

Performs all necessary steps (generating, inserting) to include a whole SPL command in the resulting grammar.

Parameters

spl_command (SPLCommand) – an SPL command to include

generate_grammar_block(name, template_block, source, targets)

Generate whole block for the grammar.

Generates a block that can be included in the grammar, based on the loaded parts from the template. Performs replacement of the placeholders defined in the gramars for the actual values

Parameters

  • name (str) – name of the command to work with

  • template_block (dict) – a template grammar block to be used

  • source (str) – a placeholder to be replaced

  • targets (list) – the actual values for replacement

generate_include(name, type)

Generate “include” line for the grammar.

Parameters

  • name (str) – name of a term

  • type (str) – type of the include, e.g. “commands”

Returns

the generated “include” line

Return type

str

save_grammar(outfile)

Save the generated grammar to the specified file.

Parameters

outfile (str) – name of the file to save the grammar into