SPL Parser Package Modules¶
This page lists documentation for the modules contained in spl_parser package.
spl_parser.cli¶
Module cli defines functionality for the CLI part of spl_parser. The CLI interface defines two operating modes (generate/view). Both operating modes can be used with a remote Splunk server or with local files.
-
validate_file
(ctx, params, file)¶ Callback to validate provided local file.
- Parameters
ctx – context object obtained from click
params – parameter object obtained from click
file (str) – name of the local file to validate
- Raises
click.BadParameter – if the provided file is of json, neither conf format
- Returns
the provided filename, if valid
- Return type
str
-
validate_url
(ctx, paarams, url)¶ Callback to validate provided URL.
Performs validation of the URL towards a regular expresssion.
- Parameters
ctx – context object obtained from click
params – parameter object obtained from click
url (str) – URL to validate
- Raises
click.BadParameter – if the provided URL has wrong format
- Returns
the provided URL, if valid
- Return type
str
spl_parser.cli_print¶
This module defines functions for easy and pretty printing of messages to terminal. This is ensured using click package.
-
enable_debug
()¶ Enable debugging.
-
log_message
(level, message)¶ Print log message to the terminal by using click.
Debugging must be enabled to print DEBUG messages.
- Parameters
level (str) – The level of the message. One of ERROR, WARNING, INFO, or DEBUG
message (str) – The message to print.
-
print_kv
(key, value)¶ Print key-value pair to the terminal.
Key is printed in bold style, whereas the value is printed normally.
- Parameters
key (str) –
value (str) –
spl_parser.exceptions¶
Defines specific exceptions used in spl_parser package.
-
exception
AuthenticationError
¶ Used when there is error in authentication towards a remote Splunk server.
-
exception
CommandNotFoundError
(message)¶ Error signalizing that command could not be retrieved.
Message should specify name of the command.
-
exception
ConnectionError
¶ Generic error which occurs in communication with a remote Splunk server.
-
exception
InitError
¶ Error occuring during initialization phase.
Such error can be related to the Pseudo-BNF grammar used for parsing, or a tmLanguage template.
-
exception
ParsingError
(message)¶ Error that occurs during parsing.
Message can specify a name of the command or file where the parsing error occurred.
-
exception
SplParserError
¶ Generic exception used with spl_parser package.
spl_parser.searchbnf_parser¶
This module defines functions that allow to parse searchbnf definitions in various fomats (JSON/conf) into their Python representations for further operations.
-
parse_conf
(conf_file, parts=[])¶ Parse searchbnf.conf file.
The provided searchbnf.conf file is processed and parsed to return all contained SPL terms. Optionally, only a subset of the file may be processed, by specifying a list containing names of the sections to be processed.
- Parameters
conf_file (str) – path to the searchbnf.conf file
parts (list, optional) – names of parts to be processed. Defaults to empty list(), meaning the whole file.
- Returns
dictionary of SPLTerms objects
- Return type
dict
-
parse_json
(json_data, parts=[])¶ Parse searchbnf data in json format.
The provided searchbnf data in json is processed and parsed to return all contained SPL terms. Optionally, only a subset of the file may be processed, by specifying a list containing names of the terms to be processed.
- Parameters
json_data (dict) – json data containing searchbnf definitions
parts (list, optional) – names of parts to be processed. Defaults to empty list(), meaning all the data.
- Returns
dictionary of SPLTerms objects
- Return type
dict
-
parse_spl_term
(name, data)¶ Parse an SPL term from the pre-loaded data.
The provided data must have a dict-like format, representing a single SPL term in a standard fomat found in searchbnf.conf or searchbnf.json files. If the provided term is a command, an SPLCommand object is created instead of SPLTerm.
- Parameters
name (str) – name of the term to parse
data (dict) – dict-like data representing a single SPL term
- Returns
an initialized SPLTerm object
- Return type
spl_parser.spl_objects¶
Module spl_objects defines SPLTerm and SPLCommand classes that serve as Python representations for SPL commands and terms.
-
class
SPLCommand
(name, syntax)¶ Object representing an SPL command.
- Parameters
name (str) – name of the SPL command
syntax (str) – syntax for the SPL command
-
parse
(syntax_trees)¶ Parse full properties of the SPL command from syntax trees.
Details for the SPL command (its arguments, functions, operators) are parsed from syntax trees. The trees related to the processed SPL command are chosen from all available trees. If the command cannot be parsed correctly, a warning is printed to the terminal.
- Parameters
syntax_trees (dict) – dictionary containing pre-built syntax trees
-
print
()¶ Pretty print the SPL command details to terminal.
-
class
SPLTerm
(name, syntax)¶ Object representing a generic SPL term.
- Parameters
name (str) – name of the term
syntax (str) – syntax for the term
-
get_bnf_syntax
()¶ Retrieve Pseudo-BNF syntax from the term representation.
- Returns
syntax string in Pseudo-BNF format
- Return type
str
-
print
()¶ Pretty print the SPL term details to terminal.
spl_parser.spl_resource¶
Module spl_resource contains definition of the workflow for the main functionalities of the package - generating the grammar and viewing commands. These functionalities are performed on objects called SplResource. Based on the operation mode, a LocalSplResource, or a RemoteSplResource is used.
-
class
LocalSplResource
(file)¶ Class defining an SPL resource based on a local file.
-
fetch_spl_terms
(spl_terms=[])¶ Fetch information for SPL terms from the resource.
- Parameters
spl_terms (list, optional) – names of SPL terms to be fetched. Defaults to empty list(), meaning all available resources.
-
-
class
RemoteSplResource
(url, username, password)¶ Class defining an SPL resource based on a remote Splunk server.
- Parameters
url (str) – URL of a remote Splunk server
username (str) – username to authenticate to the remote Splunk server
password (str) – password corresponding to the username
-
async
async_fetch_spl_term
(spl_term)¶ Asynchronously fetch details about an SPL term.
- Parameters
spl_term (SPLTerm) – SPL term to retrieve
-
async
async_fetch_spl_terms
(spl_terms)¶ Asynchronously fetch details about SPL terms.
- Parameters
spl_terms (list) – SPL terms to retrieve
-
async
async_get
(url)¶ Perform asynchronous GET request.
- Parameters
url (str) – URL to use
- Returns
JSON data received as response
- Return type
dict
-
build_url
(term_name=None)¶ Build a URL for the searchbnf file on the remote Splunk server.
If a term is specified, builds URL for the specific term, otherwise builds URL for all terms.
- Parameters
term_name (str, optional) – name of the term. Defaults to None (all terms).
- Returns
the build URL
- Return type
str
-
fetch_spl_terms
(spl_terms=[])¶ Fetch information for SPL terms from the resource.
- Parameters
spl_terms (list, optional) – names of SPL terms to be fetched. Defaults to empty list(), meaning all available resources.
-
class
SplResource
¶ Base class defining SplResource
-
fetch_spl_terms
(spl_terms=[])¶ Fetch information for SPL terms from the resource.
- Parameters
spl_terms (list, optional) – names of SPL terms to be fetched. Defaults to empty list(), meaning all available resources.
-
generate_grammar
(outfile)¶ Generate a tmLanguage grammar for SPL.
Performs all necessary steps to fetch, process, build and generate a tmLanguage grammar file for SPL. Informational messages are printed to terminal during the generation process.
- Parameters
outfile (str) – name of the file to save the generated grammar into
-
view_command
(command)¶ View details about an SPL command.
Performs all necessary steps to fetch, process and view information about the provided command. The command details and informational messages are printed to terminal.
- Parameters
command (str) – name of the command to view
- Raises
CommandNotFoundError – if the specified command was not found
-
spl_parser.syntax_parser¶
This module defines the core functions responsible for the parsing of Pseudo-BnF grammar, building syntax trees from it and manipulating with them.
-
class
LarkParser
(grammar)¶ Class serving as a simple wrapper for Lark.
- Parameters
grammar (str) – loaded contents of the file defining Lark grammar for Pseudo-BNF.
-
parse
(expression)¶ Parse a Pseudo-BNF expression using Lark.
- Parameters
expression (str) – a Pseudo-BNF expression to parse
- Returns
created syntax tree
- Return type
lark.Tree
-
build_syntax_trees
(pseudo_bnf, spl_terms)¶ Build syntax trees from Pseudo-BNF definitions.
Provided SPLTerm objects should contain Pseudo-PNF definition of the syntax. This function builds syntax trees from the syntax definitions, based on the provided Lark grammar.
- Parameters
pseudo_bnf (str) – loaded contents of the file defining Lark grammar for Pseudo-BNF.
spl_terms (dict) – dictionary containing initialized SPLTerm objects
- Raises
InitError – if an error with the Lark grammar occurs
- Returns
dictionary containing lark.Tree objects, representing syntax of the SPL terms
- Return type
dict
-
choose_trees
(names, trees)¶ Choose syntax trees based on their names.
- Parameters
names (list) – names of the trees to choose.
trees (dict) – group of available syntax trees
- Returns
a copy of tree dict containing only trees, whose names were present in the list
- Return type
dict
Find all related syntax trees to that specified by name.
Recursively search all available trees to resolve the most accurate representation of a SPL term specified by name.
- Parameters
name (str) – name of a SPL term to process
trees (dict) – available syntax trees to search
- Returns
all found related trees
- Return type
dict
-
get_token_data
(subtree)¶ Get data from the leaf of a subtree.
- Parameters
subtree (lark.Tree) – [description]
- Returns
String representation of the data contained in the first leaf of the subtree.
- Return type
str
-
parse_syntax_tree
(tree)¶ Parse a single syntax tree.
Looks for possible arguments, operators and functions contained in the tree representation of an SPL command.
- Parameters
tree (lark.Tree) – syntax tree representing SPL command
- Returns
tuple of found arguments, functions and opreators
- Return type
tuple
spl_parser.tmlanguage_generator¶
Module defining functionality related to generating a TmLanguage grammar for SPL.
-
class
TmLanguageGenerator
(template_data)¶ Class defining a TmLanguage generator.
- Parameters
template_data (str) – contents of the file defining a tmLanguage template
-
add_command
(spl_command)¶ Add SPLCommand to the grammar.
Performs all necessary steps (generating, inserting) to include a whole SPL command in the resulting grammar.
- Parameters
spl_command (SPLCommand) – an SPL command to include
-
generate_grammar_block
(name, template_block, source, targets)¶ Generate whole block for the grammar.
Generates a block that can be included in the grammar, based on the loaded parts from the template. Performs replacement of the placeholders defined in the gramars for the actual values
- Parameters
name (str) – name of the command to work with
template_block (dict) – a template grammar block to be used
source (str) – a placeholder to be replaced
targets (list) – the actual values for replacement
-
generate_include
(name, type)¶ Generate “include” line for the grammar.
- Parameters
name (str) – name of a term
type (str) – type of the include, e.g. “commands”
- Returns
the generated “include” line
- Return type
str
-
save_grammar
(outfile)¶ Save the generated grammar to the specified file.
- Parameters
outfile (str) – name of the file to save the grammar into