Version: 0.11.1-bp155.1.6
* Tue Oct 11 2022 Michael Vetter <mvetter@suse.com>
- Update to 0.11.1:
Features:
* Additional validation checks for log formats have been
added and will result in warnings. Pass -W on the
command-line to view the warnings. The following new
check have been added:
+ Each regex must have a corresponding sample log message
that it matches.
+ Each sample must be matched by only one regex.
* Added built-in support for anonymizing content. The
:write-* commands now accept an --anonymize option
and there is an anonymize() SQL function. The
anonymization process will try to replace identifying
information with random data. For example, IPv4 addresses
are replaced with addresses in the 10.0.0.0/8 range.
(This feature is mainly intended to help with providing
information to lnav support that does not have sensitive
values.)
* Added parse_url() and unparse_url() SQL functions for
parsing URLs into a JSON object and then back again. Note
that the implementation relies on libcurl which has some
limitations, like not supporting all types of schemes
(e.g. mailto:).
* Added the subsecond-field and subsecond-units log
format properties to allow for specifying a separate
field for the sub-second portion of a timestamp.
* Added a keymap for Swedish keyboards.
Breaking changes:
* The regexp_capture() table-valued-function now returns NULL
instead of an empty string for the capture_name column if
the capture is not named.
Fixes:
* Reduce the "no patterns have a capture" error to a warning
so that it doesn't block lnav from starting up.
* Some ANSI escape sequences will now be removed before testing
regexes against a log message.
* If a line in a JSON-lines log file does not start with a
{, it will now be shown as-is and will not have the JSON
parse error.
Cost of Doing Business:
* Migrated from pcre to pcre2.
* Fri Sep 02 2022 Michael Vetter <mvetter@suse.com>
- Update to 0.11.0:
Features:
* Redesigned the top status area to allow for user-specified
messages and added a second line that displays an interactive
breadcrumb bar. The top status line now shows the clock and
the remaining area displays whatever messages are inserted
into the lnav_user_notifications table. The information that
was originally on top is now in a second line and organized
as breadcrumbs. Pressing ENTER will activate the breadcrumb bar
and the left/right cursor keys can be used to select a particular
crumb while the up/down keys can select a value to switch to.
While a crumb is selected, you can also type in some text to do
a fuzzy search on the possibilities or, if the crumb represents
an array of values, enter the index to jump to.
* The pretty-print view will now show breadcrumbs that indicate the
location of the top line in the view with the prettified structure.
* Markdown files (those with a .md extension) are now rendered in the
TEXT view. The breadcrumb bar at the top will also be updated
depending on the section of the document that you are in and you
can use it to jump to different parts of the doc.
* The ":goto" command will now accept anchor links (i.e. #section-id)
as an argument when the text file being viewed has sections. You
can also specify an anchor when opening a file by appending
"#". For example, "README.md#screenshot".
* Log message comments are now treated as markdown and rendered
accordingly in the overlay. Multi-line comments are now supported
as well.
* Metadata embedded in files can now be accessed by the
"lnav_file_metadata" table. Currently, only the front-matter in
Markdown files is supported.
* Added an integration with regex101.com to make it easier to edit
log message regular expressions. Using the new "management CLI"
(activated by the -m option), a log format can be created from
a regular expression entry on regex101.com and existing patterns
can be edited.
* In the spectrogram view, the selected value range is now shown by
an overlay that includes a summary of the range and the number of
values that fall in that range. There is also a detail panel at
the bottom that shows the log-messages/DB-rows whose values are in
that range. You can then press TAB to focus on the detail view
and scroll around.
* Add initial support for pcap(3) files using tshark(1).
* SQL statement execution can now be canceled by pressing CTRL+]
(same as canceling out of a prompt).
* To make it possible to automate some operations, there is now an
"lnav_events" table that is updated when internal events occur
within lnav (e.g. opening a file, format is detected). You
can then add SQLite TRIGGERs to this table that can perform a
task by updating other tables.
* Tags can automatically be added to messages by defining a pattern
in a log format. Under a format definition, add the tag name
into the "tags" object in a format definition. The "pattern"
property specifies the regular expression to match against a line
in a file that matches the format. If a match is found, the tag
will be applied to the log message. To restrict matches to
certain files, you can add a "paths" array whose object elements
contain a "glob" property that will be matched against file names.
* Log messages can now be detected automatically via "watch
expressions". These are SQL expressions that are executed for
each log message. If the expressions evaluates to true, an
event is published to the "lnav_events" table that includes the
message contents.
* Added the "regexp_capture_into_json()" table-valued-function that
is similar to "regexp_capture()", but returns a single row with a
JSON value for each match instead of a row for each capture.
* Added a "top_meta" column to the lnav_views table that contains
metadata related to the top line in the view.
* Added a "log_opid" hidden column to all log tables that contains
the "operation ID" as specified in the log format.
* Moved the "log_format" column from the all_logs table to a hidden
column on all tables.
* Add format for UniFi gateway.
* Added a "glob" property to search tables defined in log formats
to constrain searches to log messages from files that have a
matching log_path value.
* Initial indexing of large files should be faster. Decompression
and searching for line-endings are now pipelined, so they happen
in a thread that is separate from the regular expression matcher.
* Writing to the clipboard now falls back to OSC 52 escape sequence
if none of the clipboard commands could be detected. Your
terminal software will need to support the sequence and you may
need to explicitly enable it in the terminal.
* Added the ":export-session-to " command that writes the
current session state to a file as a list of commands/SQL
statements. This script file can be executed to restore the
majority of the current state.
* Added the "echoln()" SQL function that behaves similarly to the
":echo" command, writing its first argument to the current
output.
* Added "encode()" and "decode()" SQL functions for transcoding
blobs or text values using one of the following algorithms:
base64, hex, or uri.
* In regular expressions, capture group names are now semantically
highlighted (e.g. in the capture, (?\w+), "name" would
have a unique color). Also, operations or previews that use
that regular expression will highlight the matched data with
the same color.
* Added an lnav_views_echo table that is a real SQLite table that
you can create TRIGGERs on in order to perform actions when
scrolling in a view.
* Added a "yaml_to_json()" SQL function that converts a YAML
document to the equivalent JSON.
Breaking Changes:
* Formats definitions are now checked to ensure that values have a
corresponding capture in at least one pattern.
* Added a 'language' column to the lnav_view_filters table that
specifies the language of the 'pattern' column, either 'regex'
or 'sql'.
* Timestamps that do not have a day or month are rewritten to a
full timestamp like YYYY-MM-DD HH:MM:SS.
* Removed the summary overlay at the bottom of the log view that
displayed things like "Error rate" and the time span. It doesn't
seem like anyone used it.
* Removed the "log_msg_instance" column from the logline and search
tables since it causes problems with performance.
* Search tables now search for multiple matches within a message
instead of stopping at the first hit. Each additional match is
returned as a separate row. A "match_index" column has been
added to capture the index of the match within the message.
The table regex is also compiled with the "multiline" flag enabled
so the meaning of the '^' and '$' metacharacters are changed
to match the start/end of a line instead of the start/end of
the entire message string.
* Search tables defined in formats are now constrained to only
match log messages that are in that log format instead of all
log messages. As a benefit, the search table now includes
the columns that are defined as part of the format.
* The lnav_view_filters table will treats the tuple of
(view_name, type, language, pattern) as a UNIQUE index and
will raise a conflict error on an INSERT. Use "REPLACE INTO"
instead of "INSERT INTO" to ignore conflict error.
* The types of SQL values stored as local variables in scripts
is now preserved when used as bound variables at a later point
in the script.
Fixes:
* Toggling enabled/disabled filters when there is a SQL expression
no longer causes a crash.
* Fix a crash related to long lines that are word wrapped.
* Multiple SQL statements in a SQL block of a script are now
executed instead of just the first one.
* In cases where there were many different colors on screen, some
text would be colored incorrectly.
* The pretty-print view now handles ANSI escape sequences.
* The "overstrike" convention for doing bold and underline is now
supported. (Overstrike is a character followed by a backspace
and then the same character for bold or an underscore for
underline.)
* The ":eval" command now works with searching (using the '/'
prefix).
- Remove gcc12-includes.patch fix-for-upstream-sources.patch
* Tue Jun 07 2022 Dirk Müller <dmueller@suse.com>
- add gcc12-includes.patch to fix build with gcc 12
* Fri Feb 18 2022 Marcin Bajor <marcin.bajor@gmail.com>
- Add temporary patch (fix-for-upstream-sources.patch) for the upstram issue
gh#tstack/lnav#942
* Wed Nov 03 2021 Michael Vetter <mvetter@suse.com>
- Update to 0.10.1:
Features:
* Added ":show-only-this-file" command that hides all files except the
one for the top line in the view.
* The ":write-raw-to" command now accepts a --view flag that specifies
the source view for the data to write. For example, to write the
results of a SQL query, you would pass "--view=db" to the command.
* The commands used to access the clipboard are now configured through
the "tuning" section of the configuration.
* Added an "lnav_version()" SQL function that returns the current
version string.
* Added basic support for the logfmt file format. Currently, only files
whose lines are entirely logfmt-encoded are supported. The lines
must also contain either a field named "time" or "ts" that contains
the timestamp.
* Added the "logfmt2json()" SQL function to convert a string containing
a logfmt-encoded message into a JSON object that can be operated on
more easily.
* Added the "gzip()" and "gunzip()" SQL functions to compress values
into a blob and decompress a blob into a string.
Interface changes:
* The xclip implementation for accessing the system clipboard now writes
to the "clipboard" selection instead of the "primary" selection.
* The 'query' bookmark type and y/Y hotkeys have been removed due to
performance issues and the functionality is probably rarely used.
Bug Fixes:
* The text "send-input" would show up on some terminals instead of
ignoring the escape sequence. This control sequence was only
intended to be used in the test suite.
* Remote file synchronization has been optimized a bit.
* Configuration values loaded from the ~/.lnav/configs directory
are now included in the default configuration, so they won't be
saved into the ~/.lnav/config.json user configuration file.
* Key handling in the visual filter editor will no longer swallow
certain key-presses when editing a filter.
* Scrolling performance restored in the SQL view.
* The ':redirect-to' command now works with '/dev/clipboard'
* The field overlay (opened by pressing 'p') now shows 'log_time'
for the timestamp field instead of the name defined in the format.
* The search term in the bottom status bar will now update properly
when switching views.
* The "Out-Of-Time-Order Message" overlay will be shown again.
* The tab for the "Files" panel will be highlighted in red if there
is an issue opening a file.
* Overwritten files should be reloaded again.
* The "jget()" SQL function now returns numbers with the correct type.
* The "json_contains()" SQL function now returns false if the first
argument is NULL instead of NULL.
* The local copies of remote files are now cleaned up after a couple
days of the host not being accessed.
* The initial loading and indexing phase has been optimized.
* Mon Aug 16 2021 Michael Vetter <mvetter@suse.com>
- Update to 0.10.0:
Features:
* Files on remote machines can be viewed/tailed if they are accessible
via SSH. The syntax for specifying the host and path is similar to
scp. For example, to view the files in the /var/log directory on the
machine "host1.example.org":
user@host1.example.org:/var/log
Note that you must be able to log into the machine without any
interaction.
Added the ':filter-expr' command to filter log messages based on an SQL
expression. This command allows much greater control over filtering.
* Added the ':mark-expr' command to mark log messages based on an SQL
expression. This command makes it easier to programmatically mark
log messages compared to using SQL.
* Added support for archive files, like zip, and other compression formats,
like xz, when compiled with libarchive. When one of these types of
files is detected, they are unpacked into a temporary directory and
all of the files are loaded into lnav.
* Added an 'xpath()' table-valued function for extracting values from
strings containing XML snippets.
* Added the ':prompt' command to allow for more customization of prompts.
Combined with a custom keymapping, you can now open a prompt and prefill
it with a given value. For example, a key could be bound to the
following command to open the command prompt with ":filter-in " already
filled in:
:prompt command : 'filter-in '
* Added support for the W3C Extended Log File Format with the name
"w3c_log". Similarly to the bro log format, the header is used to
determine the columns in a particular file. However, since the columns
can be different between files, the SQL table only has a well-known set
of columns and the remainder are accessible through JSON-objects stored
in columns like "cs_headers" and "sc_headers".
* Added support for the S3 Access File Format.
* To jump to the first search hit above the top line in a view, you can
press CTRL+J instead of ENTER in the search prompt. Pressing ENTER
will jump to the first hit below the current window.
* Filtering, as a whole, can be now disabled/enabled without affecting
the state of individual filters. This includes text and time-filters
(i.e. :hide-lines-before). You can enable/disable filtering by:
pressing 'f' in the filter editor UI; executing the ':toggle-filtering'
command; or by doing an UPDATE on the "filtering" column of the
"lnav_views" SQLite table.
* Themes can now include definitions for text highlights under:
/ui/theme-defs/<theme_name>/highlights
* Added a "grayscale" theme that isn't so colorful.
* Added the humanize_file_size() SQL function that converts a numeric size
to a human-friendly string.
* Added the sparkline() SQL function that returns a "sparkline" bar made
out of unicode characters. It can be used with a single value or as
an aggregator.
* Added a "log_time_msecs" hidden column to the log tables that returns
the timestamp as the number of milliseconds from the epoch.
* Added an "lnav_top_file()" SQL function that can be used to get the
name of the top line in the top view or NULL if the line did not come
from a file.
* Added a "mimetype" column to the lnav_file table that returns a guess as
to the MIME type of the file contents.
* Added a "content" hidden column to the lnav_file table that can be used
to read the contents of the file. The contents can then be passed to
functions that operate on XML/JSON data, like xpath() or json_tree().
* Added an "lnav_top_view" SQL VIEW that returns the row for the top view
in the lnav_views table.
* The "generate_series()" SQLite extension is now included by default.
One change from the standard implementation is that both the start and
stop are required parameters.
* Added the ";.read" SQL command for executing a plain SQL file.
* Added the "-N" flag so that lnav will run without opening the default
syslog file.
Interface Changes:
* When copying log lines, the file name and time offset will be included
in the copy if they are enabled.
* Log messages that cannot be parsed properly will be given an "invalid"
log level and the invalid portions colored yellow.
* The range_start and range_stop values of the regexp_capture() results
now start at 1 instead of zero to match with what the other SQL string
functions expect.
* The ":write-cols-to" command has been renamed to ":write-table-to".
* The DB view will limit the maximum column width to 120 characters.
* The ":echo" command now evaluates its message to do variable
substitution.
* The ":write-raw-to" command has been changed to write the original
log file content of marked lines. For example, when viewing a JSON
log, the JSON-Line values from the log file will be written to the
output file. The ":write-view-to" command has been added to perform
the previous work of ":write-raw-to" where the raw content of the view
is written to the file.
Fixes:
* Unicode text can now be entered in prompts.
* The replicate() SQL function would cause a crash if the number of
replications was zero.
* Many internal improvements.
* Fri Sep 25 2020 Michael Vetter <mvetter@suse.com>
- Update to 0.9.0:
Features:
* Added support for themes and included a few as well: default, eldar,
monocai, night-owl, solarized-light, and solarized-dark. The theme
can be changed using the ':config' command, like so:
:config /ui/theme night-owl
Consult the online documentation for defining a new theme at:
https://lnav.readthedocs.io/en/latest/config.html#theme-definitions[1]
* Added support for custom keymaps and included the following: de, fr,
uk, us. The keymap can be changed using the ':config' command, like so:
:config /ui/keymap uk
Consult the online documentation for defining a new keymap at:
https://lnav.readthedocs.io/en/latest/config.html#keymap-definitions[2]
* The following JSON-Schemas have been published for the log format and
configuration JSON files:
https://lnav.org/schemas/format-v1.schema.json[3]
https://lnav.org/schemas/config-v1.schema.json[4]
Formats should be updated to reference the schema using the "$schema"
property.
* Indexing of new data in log files can now be paused by pressing '='
and unpaused by pressing it again. The bottom status bar will display
'Paused' in the right corner while paused.
* CMake is now a supported way to build.
* When viewing data from the standard-input, a symbolic name can be used
to preserve session state. The name can be changed using the
"rename-stdin" lnav script or by doing an UPDATE to the filepath
column of the lnav_file table. For example, to assign the name
"journald", the following SQL statement can be executed in lnav:
;UPDATE lnav_file SET filepath='journald' WHERE filepath='stdin'
* The size of the terminal can be accessed in SQL using the $LINES and
$COLS variables.
* The raise_error(msg) SQL function has been added to make it easier to
raise an error in an lnav script to stop execution and notify the user.
* Added the json_concat() function to make it easier to append/concatenate values onto arrays.
* Added the ":write-jsonlines-to" command that writes the result of a SQL query to a file in the JSON Lines format.
Interface Changes:
* Data piped into lnav is no longer dumped to the console after exit.
Instead a file containing the data is left in .lnav/stdin-captures/
and a message is printed to the console indicating the file name.
* In time-offset mode, the deltas for messages before the first mark
are now negative instead of relative to the start of the log.
* The $XDG_CONFIG_HOME environment variable (or ~/.config directory) are
now respected for storing lnav's configuration. If you have an existing
~/.lnav directory, that will continue to be used until you move it to
$XDG_CONFIG_HOME/lnav or ~/.config/lnav.
* Removed the ':save-config' command. Changes to the configuration are now
immediately saved.
Fixes:
* Added 'notice' log level.
* If a "timestamp-format" is used in an element of a "line-format", the
field name is ignored and a formatted timestamp is always used.
* Ignore stdin when it is connected to /dev/null.