All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- Added
--include-libpostalto includelibpostaloutput alongside another geocoder's output.
- If we find a
\0byte in cached data, try looking it up again. Also, report an error if the Smarty API ever returns a\0byte.
- Roll back
serde_deriveto version 1.0.171 and pin it there, to avoid using the newserde_derivethat uses a pre-built binary compiler plugin. The binary plugin has not been successfully reproduced from source when I last looked, and it represents an unacceptable security risk, since it can't be audited without disassembling the binary code. We'll need a longer-term plan for this.
- In server mode, "prime" libpostal so that it doesn't take forever to process the first request.
- Fix build by ignoring test that requires libpostal data.
geocode-csv serverwill now start a basic, experimental REST server.- The
--cache-hits-onlyflag can be used to bypass geocoding for a given backend where we're only interested normalizing addresses we already know about.
- Use a
Justfileto document and automate common maintainer tasks.
- Fixed an issue where libpostal would occasionally parse cities as suburbs. Thank you, Tom, for the PR!
- Skip records with empty streets after normalization.
- Fix broken release build caused by incorrect use of
cfg!macro.
- Added lots of debugging assertions to help detect potential memory overflows. None of these were the actual cause of the recent bugs, but the ones that we left in were nice to have.
- Fixed hang/memory overflow where we forwarded
&[Address]bocks with no addresses to the cache layer. - Updated to latest
crossand installedprotocinside the Docker container.
- Update all dependencies to see if we can get a clean build with no security advisories. None of the advisories looked like they should affect normal usage, but it's better to fix them.
- Do not attempt to geocode empty addresses. This has recently started causing SmartyStreets to return an error.
- The build process now requires at least
cmake 3.12or so, which breaks Ubuntu 18.04 unless you use a PPA. - We now require
protobuf-compilerto build.
- Update dependencies to fix minor security advisories.
- Updated a bunch of dependencies, including BigTable client used for caching.
- Retries now use exponential backoff. This means that we should now give up after about 30 seconds, instead of 10 or 12 like before.
--max-retriescan now be used to control how many times we retry before giving up.
--max-addresses-per-secondallows specifying a rate limit for any external geocoder. This is particularly useful with services like Smarty that impose their own rate limits.
- Report metric
geocodecsv.selected_errors.count, which breaks down particularly interesting errors bycomponentandcause. We focus on reporting this information for remote APIs, currently including BigTable and Smarty. This is meant to supplement the existinggeocodecsv.chunks_retried.totalandgeocodecsv.chunks_failed.totalmetrics, which already report generic error statistics, but which can't say what failed or why.
- Renabled Mac M1 builds after fixing cross-compilation of
libpostal.
- Disabled Mac M1 builds until we get cross-compilation working with
libpostal. This will allow us to test other platforms.
- Renamed
--cache-record-keysto--cache-output-keys.
- Cleaned up packaging, build and CI.
Internal-only releases identified as either 1.0.2 or 2.0.0-alpha.1. No official binaries were ever built, and this exact release does not exist in git history. Included for completeness.
- Support for "geocoding" using
libpostal. This returns parsed and normalized address fields, but no lat/lon data. To use this, pass--geocoder=libpostal. - Optionally normalize addresses using
libpostalwhen using other geocoders. - We can now cache data using either Redis or BigTable.
- You can use
--cache-record-keysto output the cache keys.
- You can use
- We now calculate extensive geocoding metrics, and can either print them to standard output, or send them to NewRelic.
SMARTY_AUTH_IDis now preferred overSMARTYSTREETS_AUTH_ID(though both will be supported).SMARTY_AUTH_TOKENis now preferred overSMARTYSTREETS_AUTH_TOKEN(though both will be supported).--smarty-licenseis now preferred over--license(though both will be supported).
- New naming convention for release ZIP files.
- New binary builds for ARM/M1 Macs.
- The downloadable
*.zipfiles now include both the CPU and the OS in the name. Downloading scripts will need to be adjusted. - We now use
rustlsinstead of OpenSSL internally. This shouldn't change anything, but it's a significant change.
- Restored missing binaries for existing platforms by switching to GitHub CI.
Bumping number to v1.0.0 because this has been running fine in production for quite.
- Support
--match enhanced. This only works for appropriate SmartyStreets plans.
- Updated dependencies to fix several CVEs reported by
cargo deny. I do not believe that any of these CVEs actually affectedgeocode-csvin practice, but better safe than sorry.
- Updated to latest
tokioandhyperlibraries. This represents a major change to some of our core libraries, but it means that we're finally on stabletokioand not a pre-release. - Replaced error-reporting code, so some error output might look different.
This build has released binaries, but the version number wasn't updated, at it still claims to be v0.3.0-beta.3. Since the code is identical, this isn't the end of the world.
- Fixed Travis CI build.
- Rebuild with modern Rust toolchain.
- Added a
--licenseoption to enable use of rooftop geocoding.
- Fixed a number of security advisories in supporting libraries. None of these appear to have been easily exploitable using an invalid CSV file as input.