Skip to content

Commit

Permalink
Merge pull request tonerdo#44 from rogusdev/quoted-comments-and-more
Browse files Browse the repository at this point in the history 
Quoted comments and more
  • Loading branch information
@rogusdev
rogusdev authored Dec 16, 2020
2 parents 142452a + 4e59f9c commit faf6518
Show file tree
Hide file tree
Showing 22 changed files with 1,473 additions and 458 deletions.
179 changes: 114 additions & 65 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,110 +73,151 @@ You can also pass a `LoadOptions` object arg to all `DotNetEnv.Env.Load` variant

```csharp
new DotNetEnv.Env.LoadOptions(
trimWhitespace: false,
isEmbeddedHashComment: false,
unescapeQuotedValues: false,
clobberExistingVars: false,
parseVariables: false
setEnvVars: true,
clobberExistingVars: true
)
```

All parameters default to true, which means:

1. `trimWhitespace`, first arg: `true` in order to trim
leading and trailing whitespace from keys and values such that
1. `setEnvVars`, first arg: `true` in order to actually update env vars.
Setting it `false` allows consumers of this library to process the .env file
but use it for things other than updating env vars, as a generic configuration file.
The Load methods all return an `IEnumerable<KeyValuePair<string,string>> for this.

```env
KEY = value
```

Would then be available as
```csharp
"value" == System.Environment.GetEnvironmentVariable("KEY")
null == System.Environment.GetEnvironmentVariable(" KEY ")
KEY=value
```

`false` would mean:
```csharp
" value" == System.Environment.GetEnvironmentVariable(" KEY ")
var kvps = DotNetEnv.Env.Load(
new DotNetEnv.Env.LoadOptions(
setEnvVars: false
)
)
// not "value" from the .env file
null == System.Environment.GetEnvironmentVariable("KEY")
"KEY" == kvps.First().Key
"value" == kvps.First().Value
```

2. `isEmbeddedHashComment`, second arg: `true` in order to allow inline comments
2. `clobberExistingVars`, second arg: `true` to always set env vars,
`false` would leave existing env vars alone.

```env
KEY=value # comment
KEY=value
```

Would then be available as
```csharp
"value" == System.Environment.GetEnvironmentVariable("KEY")
System.Environment.SetEnvironmentVariable("KEY", "really important value, don't overwrite");
DotNetEnv.Env.Load(
new DotNetEnv.Env.LoadOptions(
clobberExistingVars: false
)
)
// not "value" from the .env file
"really important value, don't overwrite" == System.Environment.GetEnvironmentVariable("KEY")
```

`false` would mean:
```csharp
"value # comment" == System.Environment.GetEnvironmentVariable("KEY")
## .env file structure

All lines must be valid assignments or empty lines (with optional comments).

A minimal valid assignment looks like:
```
KEY=value
```

Which is most useful when you want to do something like:
```env
KEY=value#moreValue#otherValue#etc
There can optionally be one of a few export or equivalent keywords at the beginning
and there can be a comment at the end, values can be quoted to include whitespace,
and interpolated references can be included (unquoted values as well as double quoted,
with optional braces in both cases -- but often more useful in unquoted), like:
```
export KEY="extra $ENVVAR value" # comment
set KEY2=extra${ENVVAR}value # comment
```

3. `unescapeQuotedValues`, third arg: `true` in order to unescape/parse
quoted (single or double) values as being strings with escaped chars
such as newline ("\n"), but also handles unicode chars
(e.g. "\u00ae" and "\U0001F680") -- note that you can always include
unescaped unicode chars anyway (e.g. "日本") if your .env is in UTF-8.
Also note that there is no need to escape quotes inside.
The options for the export keyword are:

```env
KEY="quoted\n\tvalue"
```
export # bash
set # windows cmd
SET # windows cmd
set -x # fish

Would then be available as
```csharp
"quoted
value" == System.Environment.GetEnvironmentVariable("KEY")
This allows the `.env` file itself to be `source`-d like `. .env`
to load the env vars into a terminal session directly.

The options for quoting values are:

1. `""` double: can have everything: interpolated variables, plus whitespace, escaped chars, and byte code chars
1. `''` single: can have whitespace, but no interpolation, no escaped chars, no byte code chars -- notably not even escaped single quotes inside -- single quoted values are for when you want truly raw values
1. unquoted: can have interpolated variables, but no whitespace, and no escaped chars, nor byte code chars.

As these are the options bash recognizes. However, while bash does have
special meaning for each of these, in this library, they are all the same,
other than that you do not need to escape single quote chars inside
a double quoted value, nor double quotes inside single quotes.

As a special note: if a value is unquoted, it can still include a `#` char,
which might look like it is starting a comment, like:
```
KEY=value#notcomment #actualcomment
```

`false` would mean:
```csharp
"\"quoted\\n\\tvalue\"" == System.Environment.GetEnvironmentVariable("KEY")
This is how bash works as well:
```
export TEST=value#notcomment #actualcomment
env | grep TEST
# TEST=value#notcomment
```

4. `clobberExistingVars`, fourth arg: `false` to avoid overwriting existing environment variables
You can also declare unicode chars as byte codes in double quoted values:

```env
KEY=value
```
UTF8 btes: "\xF0\x9F\x9A\x80" # rocket 🚀
UTF16 bytes: "\uae" # registered ®
UTF32 bytes: "\U1F680" # rocket 🚀

```csharp
System.Environment.SetEnvironmentVariable("KEY", "really important value, don't overwrite");
DotNetEnv.Env.Load(
new DotNetEnv.Env.LoadOptions(
clobberExistingVars: false
)
)
"really important value, don't overwrite" == System.Environment.GetEnvironmentVariable("KEY") // not "value" from the .env file
Capitalization on the hex chars is irrelevant, and leading zeroes are optional.

And standard escaped chars like `\t`, `\\``, `\n`, etc are also recognized
-- though quoted strings can also be multi line, e.g.:

```
KEY="value
and more"
OTHER='#not_comment
line2'
```

5. `parseVariables`, fifth arg: `true` to parse existing environment variables
Loaded gives:
```
"value\nand more" == System.Environment.GetEnvironmentVariable("KEY")
"#not_comment\nline2" == System.Environment.GetEnvironmentVariable("OTHER")
```

```env
FIRST_KEY=value1
SECOND_KEY=value2and$FIRST_KEY
THIRD_KEY=$EXISTING_ENVIRONMENT_VARIABLE;andvalue3
You can also include whitespace before and after the equals sign in assignments,
between the name/identifier, and the value, quoted or unquoted.
Note that the pre/trailing and post/leading whitespace will be ignored.
If you want leading whitepace on your values, quote them with whitespace.
```
WHITE_BOTH = value
WHITE_QUOTED=" value "
```

Would then be available as
```csharp
"value1" == System.Environment.GetEnvironmentVariable("FIRST_KEY")
"value2andvalue1" == System.Environment.GetEnvironmentVariable("SECOND_KEY")
"value;andvalue3" == System.Environment.GetEnvironmentVariable("THIRD_KEY") //EXISTING_ENVIRONMENT_VARIABLE already set to "value"
Loaded gives:
```
"value" == System.Environment.GetEnvironmentVariable("WHITE_BOTH")
" value " == System.Environment.GetEnvironmentVariable("WHITE_QUOTED")
```

Note that bash env vars do not allow white space pre or post equals,
so this is a convenience feature that will break sourcing .env files.
But then, not all of this is 100% compatible anyway, and that's ok.

## A Note about Production and the Purpose of this library
Note that other .env parsing libraries also might have slightly different rules
-- no consistent rules have arisen industry wide yet.

## A Note about Production and the Purpose of This Library

You should not be using a .env file in production. The purpose of this library is to enable easy local development.

Expand All @@ -199,8 +240,16 @@ If you have found a bug or if you have a feature request, please report them at

Run `dotnet test test/DotNetEnv.Tests` to run all tests.

Or some more specific test examples:

dotnet test --filter "FullyQualifiedName~DotNetEnv.Tests.EnvTests.BadSyntaxTest"
dotnet test --filter "FullyQualifiedName~DotNetEnv.Tests.ParserTests.ParseAssignment"

`src/DotNetEnvEnv/Env.cs` is the entry point for all behavior.

`src/DotNetEnvEnv/Parsers.cs` defines all the [Sprache](https://github.com/sprache/Sprache) parsers.


Open a PR on Github if you have some changes, or an issue if you want to discuss some proposed changes before creating a PR for them.

## License
Expand Down
3 changes: 3 additions & 0 deletions src/DotNetEnv/AssmblyInfo.cs
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
using System.Runtime.CompilerServices;

[assembly: InternalsVisibleTo("DotNetEnv.Tests")]
10 changes: 7 additions & 3 deletions src/DotNetEnv/DotNetEnv.csproj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
<Title>DotNetEnv</Title>
<Description>A .NET library to load environment variables from .env files</Description>
<AssemblyTitle>DotNetEnv</AssemblyTitle>
<AssemblyVersion>1.4.0</AssemblyVersion>
<PackageVersion>1.4.0</PackageVersion>
<AssemblyVersion>2.0.0</AssemblyVersion>
<PackageVersion>2.0.0</PackageVersion>
<TargetFramework>netstandard1.3</TargetFramework>
<AssemblyName>DotNetEnv</AssemblyName>
<OutputType>Library</OutputType>
Expand All @@ -17,6 +17,10 @@
</PropertyGroup>

<ItemGroup>
<None Include="../../LICENSE" Pack="true" PackagePath=""/>
<None Include="../../LICENSE" Pack="true" PackagePath="" />
</ItemGroup>

<ItemGroup>
<PackageReference Include="Sprache" Version="2.3.1" />
</ItemGroup>
</Project>
89 changes: 48 additions & 41 deletions src/DotNetEnv/Env.cs
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
using System;
using System.Linq;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using System.Text;

namespace DotNetEnv
{
Expand All @@ -11,78 +13,83 @@ public class Env

private static LoadOptions DEFAULT_OPTIONS = new LoadOptions();

public static void Load(string[] lines, LoadOptions options = null)
public static IEnumerable<KeyValuePair<string, string>> Load (string[] lines, LoadOptions options = null)
{
if (options == null) options = DEFAULT_OPTIONS;

Vars envFile = Parser.Parse(
lines,
options.TrimWhitespace,
options.IsEmbeddedHashComment,
options.UnescapeQuotedValues,
options.ParseVariables
);
LoadVars.SetEnvironmentVariables(envFile, options.ClobberExistingVars);
return LoadContents(String.Join("\n", lines), options);
}

public static void Load(string path, LoadOptions options = null)
public static IEnumerable<KeyValuePair<string, string>> Load (string path, LoadOptions options = null)
{
if (!File.Exists(path)) return;
Load(File.ReadAllLines(path), options);
// in production, there should be no .env file, so this should be the common code path
if (!File.Exists(path))
{
return Enumerable.Empty<KeyValuePair<string, string>>();
}
return LoadContents(File.ReadAllText(path), options);
}

public static void Load(Stream file, LoadOptions options = null)
public static IEnumerable<KeyValuePair<string, string>> Load (Stream file, LoadOptions options = null)
{
var lines = new List<string>();
var currentLine = "";
using (var reader = new StreamReader(file))
{
while (currentLine != null)
return LoadContents(reader.ReadToEnd(), options);
}
}

public static IEnumerable<KeyValuePair<string, string>> LoadContents (string contents, LoadOptions options = null)
{
if (options == null) options = DEFAULT_OPTIONS;

if (options.SetEnvVars)
{
if (options.ClobberExistingVars)
{
return Parsers.ParseDotenvFile(contents, Parsers.SetEnvVar);
}
else
{
currentLine = reader.ReadLine();
if (currentLine != null) lines.Add(currentLine);
return Parsers.ParseDotenvFile(contents, Parsers.NoClobberSetEnvVar);
}
}
Load(lines.ToArray(), options);
else
{
return Parsers.ParseDotenvFile(contents, Parsers.DoNotSetEnvVar);
}
}

public static void Load(LoadOptions options = null) =>
public static IEnumerable<KeyValuePair<string, string>> Load (LoadOptions options = null) =>
Load(Path.Combine(Directory.GetCurrentDirectory(), DEFAULT_ENVFILENAME), options);

public static string GetString(string key, string fallback = default(string)) =>
public static string GetString (string key, string fallback = default(string)) =>
Environment.GetEnvironmentVariable(key) ?? fallback;

public static bool GetBool(string key, bool fallback = default(bool)) =>
public static bool GetBool (string key, bool fallback = default(bool)) =>
bool.TryParse(Environment.GetEnvironmentVariable(key), out var value) ? value : fallback;

public static int GetInt(string key, int fallback = default(int)) =>
public static int GetInt (string key, int fallback = default(int)) =>
int.TryParse(Environment.GetEnvironmentVariable(key), out var value) ? value : fallback;

public static double GetDouble(string key, double fallback = default(double)) =>
public static double GetDouble (string key, double fallback = default(double)) =>
double.TryParse(Environment.GetEnvironmentVariable(key), NumberStyles.Any, CultureInfo.InvariantCulture, out var value) ? value : fallback;

public class LoadOptions
{
public bool TrimWhitespace { get; }
public bool IsEmbeddedHashComment { get; }
public bool UnescapeQuotedValues { get; }
public bool SetEnvVars { get; }
public bool ClobberExistingVars { get; }
public bool ParseVariables { get; }

public LoadOptions(
bool trimWhitespace = true,
bool isEmbeddedHashComment = true,
bool unescapeQuotedValues = true,
bool clobberExistingVars = true,
bool parseVariables = true
)
{
TrimWhitespace = trimWhitespace;
IsEmbeddedHashComment = isEmbeddedHashComment;
UnescapeQuotedValues = unescapeQuotedValues;
bool setEnvVars = true,
bool clobberExistingVars = true
) {
SetEnvVars = setEnvVars;
ClobberExistingVars = clobberExistingVars;
ParseVariables = parseVariables;
}
}
}

public static class Extensions
{
public static Dictionary<string, string> ToDictionary (this IEnumerable<KeyValuePair<string, string>> kvps) =>
kvps.GroupBy(kv => kv.Key).ToDictionary(g => g.Key, g => g.Last().Value);
}
}
Loading

0 comments on commit faf6518

Please sign in to comment.