NAME
Config::JSON::Enhanced - JSON-based config with C/Shell-style comments,
verbatim sections and variable substitutions
VERSION
Version 0.03
SYNOPSIS
This module provides subroutine config2perl() for parsing configuration
content, from files or strings, based on, what I call, "enhanced JSON"
(see section "ENHANCED JSON FORMAT" for more details). Briefly, it is
standard JSON which allows:
* C-style, C++-style or shell-style comments.
* template variables (e.g. <% appdir %>) which are substituted with
user-specified data.
* verbatim sections which are a sort of here-doc for JSON which may
be spanning multiple lines and contained single and double quotes are
not required to be escaped. This enhances the readbility of long JSON
which may contain, in my case, long shell scripts with lots of quotes
and newlines.
It has been tested with unicode data (see
t/25-config2perl-complex-utf8.t) with success. But who knows ?!?!
Here is an example:
use Config::JSON::Enhanced qw/config2perl/;
# simple "enhanced" JSON with comments in 3 styles: C,shell,CPP
my $configdata = <<'EOJ';
{
/* 'a' is ... */
"a" : "abc",
# b is ...
"b" : [1,2,3],
"c" : 12 // c is ...
}
EOJ
my $perldata = config2perl({
'string' => $configdata,
'commentstyle' => "C,shell,CPP",
});
die "call to config2perl() has failed" unless defined $perldata;
# the standard JSON:
# {"a" : "abc","b" : [1,2,3], "c" : 12}
# this "enhanced" JSON demonstrates "verbatim sections"
# the puprose of which is to make more readable JSON strings
# by allowing them spanning over multiple lines
# and no need for escaping double quotes.
my $configdata = <<'EOJ';
{
"a" : <%begin-verbatim-section%>
This is a multiline
string
"quoted text" and 'quoted like this also'
will be retained in the string escaped.
Comments like /* this */ or # this comment
will be removed.
White space from beginning and end will be chomped.
<%end-verbatim-section%>
,
"b" = 123
}
EOJ
my $perldata = config2perl({
'string' => $configdata,
'commentstyle' => "C,shell,CPP",
});
die "call to config2perl() has failed" unless defined $perldata;
# the standard JSON (notice that "a" value is in a single line,
# here printed broken for readability):
# {"a" :
# "This is a multiline\nstring\n\"quoted text\" and 'quoted like
# this also'\nwill be retained in the string escaped.\nComments
# like or # this comment\nwill be removed.\nWhite space from
# beginning and end will be chomped.",
# "b" : 123
# };
# this "enhanced" JSON demonstrates the use of variables
# which will be substituted during the transformation to
# standard JSON with user-specified data.
my $configdata = <<'EOJ';
{
"d" : [1,2,<% tempvar0 %>],
"configfile" : "<%SCRIPTDIR%>/config/myapp.conf",
"username" : "<% username %>"
}
}
EOJ
my $perldata = config2perl({
'string' => $configdata,
'commentstyle' => "C,shell,CPP",
# user-specified data to replace the variables in
# the "enhanced" JSON above:
'variable-substitutions' => {
'tempvar0' => 42,
'username' => getlogin(),
'SCRIPTDIR' => $FindBin::Bin,
},
});
die "call to config2perl() has failed" unless defined $perldata;
# the standard JSON
# (notice how all variables in <%...%> are now replaced):
# {"d" : [1,2,42],
# "username" : "yossarian",
# "configfile" : "/home/yossarian/B52/config/myapp.conf"
# }
EXPORT
* config2perl
SUBROUTINES
config2perl
my $ret = config2perl($params);
Arguments:
* $params
Return value:
* $ret on success or undef on failure.
Given Enhanced JSON content it removes any comments, it replaces all
template strings, if any, and then parses what remains as standard JSON
into a Perl data structure which is returned.
JSON content to-be-parsed can be specified with one of the following
keys in the input parameters hashref ($params):
* filename : content is read from a file with this name.
* filehandle : content is read from a file which has already been
opened for reading by the caller.
* string : content is contained in this string.
Additionally, input parameters can contain the following keys:
* commentstyle : specify what comment style(s) to be expected in the
input content (if any) as a comma-separated string. For example
'C,CPP,shell,custom(<<)(>),custom(REM)()'>. These are the values it
understands:
* C : comments take the form of C-style comments which are
exclusively within /* and */. For example * I am a comment */. This
is the default comment style if none specified.
* CPP : comments can the the form of C++-style comments which are
within /* and */ or after // until the end of line. For example /*
I am a comment */, // I am a comment to the end of line.
* shell : comments can be after # until the end of line. For
example, # I am a comment to the end of line.
* custom : comments are enclosed (or preceded) by custom,
user-specified strings. The form is custom(OPENING)(CLOSING).
OPENING is required and CLOSING is optional meaning that the
comment extends to the end of line (just like shell comments). For
example custom(<<)(>>) or custom({{)(}) or custom(REM)() or
custom(<<<<)(>>). OPENING and CLOSING do not need to be of the same
character length as it is obvious from the previous example.
* variable-substitutions : a hashref whose keys are variable names as
they occur in the input Enhanced JSON content and their corresponding
values should substitute them. Enhanced JSON, can contain template
variables in the form <% my-var-1 %>. These must be replaced with
data which is supplied to the call of config2perl() under the
parameters key variable-substitutions, for example:
config2perl({
"variable-substitutions" => {
"my-var-1" => 42,
"SCRIPTDIR" => "/home/abc",
},
"string" => '{"a":"<% my-var-1 %>", "b":"<% SCRIPTDIR %>/app.conf"}',
});
Variable substitution will be performed in both keys and values of
the input JSON.
* remove-comments-in-strings : by default JSON strings (both keys and
values) are keeping all their content, even if it looks like it
contains comments which they are usually removed elsewhere. By
setting this flag to 1, anything that looks like a comment we must
understand (as per the parameter commentstyle). For example consider
the JSON string "hello/*a comment*/" (which can be a key or a value).
If remove-comments-in-strings is set to 1, then the JSON string will
become hello. If set to 0 (which is the default) it will be
unchanged.
* tags : specify the opening and closing tags for template variables
and verbatim section as an ARRAYref of exactly 2 items (the opening
and the closing tags). By default the opening tag is >% and the
closing tag is %<. By setting tags = [ '[::', '::]' ]> you must
declare templated variables like this: {:: var1 ::] and verbatim
sections like this: [:: begin-verbatim-section ::].
* debug : set this to a positive integer to increase verbosity and
dump debugging messages. Default is zero for zero verbosity.
See section "ENHANCED JSON FORMAT" for details on the format of what I
call enhanced JSON.
config2perl returns the parsed content as a Perl data structure on
success or undef on failure.
ENHANCED JSON FORMAT
This is JSON with added reasonable, yet completely ad-hoc, enhancements
(from my point of view).
These enhancements are:
* Comments are allowed:
* C-style comments take the form of C-style comments which are
exclusively within /* and */. For example * I am a comment */
* C++-style comments can the the form of C++-style comments which
are within /* and */ or after // until the end of line. For example
/* I am a comment */, // I am a comment to the end of line.
* shell-style comments can be after # until the end of line. For
example, # I am a comment to the end of line.
* Template variables support : template variables in the form of
<%HOMEDIR%> will be substituded with their actual values specified by
the user during parsing. Note that variable names are case sensitive,
they can contain spaces, hyphens etc., for example: <% abc- 123 - xyz
%> (the variable name is abc- 123 - xyz (notice the multiple spaces
between 123 and xyz).
The tags for denoting a template variable are controled by the 'tags'
parameter to the sub config2perl. <% and %> are the defaults.
* Verbatim Sections : similar to here-doc, this feature allows for
string values to span over multiple lines and to contain un-escpaed
quotes. This is useful if you want a JSON value to contain a shell
script, for example.
* Unfortunately, there is not support for ignoring superfluous commas
in JSON, in the manner of glorious Perl.
Verbatim Sections
A Verbaitm Section in this ad-hoc, so-called Enhanced JSON is content
enclosed within <%begin-verbatim-section%> and <%end-verbatim-section%>
tags. A verbatim section's content may span multiple lines (which when
converted to JSON will preserve them by escaping), can contain comments
(see the beginning of this section) and can contain template variables
to be substituted with user-specified data.
The content of Verbatim Sections will have all its template variables
substituted. But its comments will be left untouched.
The tags for denoting the opening and closing a verbatim section are
controled by the 'tags' parameter to the sub config2perl. <% and %> are
the defaults.
Here is an example of enhanced JSON which contains comments, a verbatim
section and template variables:
my $con = <<'EOC';
{
"long bash script" : ["/usr/bin/bash",
/* This is a verbatim section */
<%begin-verbatim-section%>
# save current dir
pushd . &> /dev/null
echo "My 'appdir' is \"<%appdir%>\""
echo "My current dir: " $(echo $PWD) " and bye"
# go back to initial dir
popd &> /dev/null
<%end-verbatim-section%>
/* the end of the verbatim section */
],
// this is an example of a template variable
"expected result" : "<% expected-res123 %>"
}
EOC
# Which, can be processed thusly:
my $res = config2perl({
'string' => $con,
'commentstyle' => 'C,CPP',
'variable-substitutions' => {
'appdir' => Cwd::abs_path($FindBin::Bin),
'expected-res123' => 42
},
});
die "call to config2perl() has failed" unless defined $res;
# following is the dump of $res, note the breaking of the lines
# in the 'long bash script' is just for readability.
# In reality, it is one long line:
{
"expected result" => 42,
"long bash script" => [
"/usr/bin/bash",
"# save current dir\npushd . &> /dev/null\necho \"My 'appdir'
is \\\"/home/andreas/PROJECTS/CPAN/Config-JSON-Enhanced/t\\\"\"\necho
\"My current dir: \" \$(echo \$PWD) \" and bye\"\n# go back to
initial dir\npopd &> /dev/null"
]
};
A JSON string can contain comments which you may want to retain. For
example if the content is a unix shell script it is possible to contain
comments like # comment. These will be removed along with all other
comments in the entire JSON input if you are using shell style
comments. Another problem is when JSON string contains comment opening
or closing strings. For example consider this cron spec : */15 * * * *
which contains the closing string of a C-style comment and will mess
everything.
You have two options in order to retain these comments:
* Set 'remove-comments-in-strings' parameter to sub config2perl to 0.
This will keep ALL comments in all strings (both keys and values).
This is a one-size-fits-all solution and it is not ideal.
* The best solution is to change the comment style of the input, so
called Enhanced, JSON to something different to the comments you are
trying to keep in your strings. So, for example, if you want to
retain the comments in a unix shell script then use C as the comment
style for the JSON. Note that it is possible (since version 0.03) to
use custom comments for the JSON. This greatly increases your chances
to make config2perl understand what comments you want to keep as part
of your data.
TIPS
You can change the tags used in denoting the template variables and
verbatim sections with the tags parameter to the sub config2perl. Use
this feature to change tags to something else if your JSON contains the
same character sequence for these tags and avoid clashes and unexpected
substitutions. <% and %> are the default tags.
Similarly, custom comment style can be employed if your JSON strings
contain something that looks like comments and you want to avoid their
removal.
WARNINGS/CAVEATS
In order to remove comments within strings, a simplistic regular
expression for extracting quoted strings is employed. It finds anything
within two double quotes. It tries to handle escaped quotes within
quoted strings. This may be buggy or may not be complex enough to
handle all corner cases. Therefore, it is possible that setting
parameter remove-comments-in-strings to 1 to sub config2perl to cause
unexpected results. Please report these cases, see SUPPORT.
AUTHOR
Andreas Hadjiprocopis, <bliako at cpan.org>
BUGS
Please report any bugs or feature requests to bug-config-json-enhanced
at rt.cpan.org, or through the web interface at
https://rt.cpan.org/NoAuth/ReportBug.html?Queue=Config-JSON-Enhanced. I
will be notified, and then you'll automatically be notified of progress
on your bug as I make changes.
SUPPORT
You can find documentation for this module with the perldoc command.
perldoc Config::JSON::Enhanced
You can also look for information at:
* RT: CPAN's request tracker (report bugs here)
https://rt.cpan.org/NoAuth/Bugs.html?Dist=Config-JSON-Enhanced
* CPAN Ratings
https://cpanratings.perl.org/d/Config-JSON-Enhanced
* Search CPAN
https://metacpan.org/release/Config-JSON-Enhanced
ACKNOWLEDGEMENTS
LICENSE AND COPYRIGHT
This software is Copyright (c) 2023 by Andreas Hadjiprocopis.
This is free software, licensed under:
The Artistic License 2.0 (GPL Compatible)