Motivation:
The current state of Swift does not leave room for library evolution of enum types used for `Error`.
To avoid having to increment Major SemVer to add a new error case that might be needed to fix a bug, the `enum-like struct` idiom should be used.
Ideally this idiom will disappear, when Swift provides a way for Swift Packages to have a "library evolution" capability.
See https://forums.swift.org/t/extensible-enumerations-for-non-resilient-libraries/35900
Modifications:
- Change: `RedisClientError` to be struct with private enum value
Result:
Should new error cases be necessary to add, they can be in Minor SemVer releases, rather than Major SemVer.
Motivation:
The TTL and PTTL commands are missing.
Modifications:
- Add TTL and PTTL commands
- Add integration tests
Result:
- Users can query the ttl in seconds or milliseconds of a key
Motivation:
The EXISTS command was missing.
Modifications:
- Add 'EXISTS' to basic commands
- Add integration tests
Result:
The existence of a key can be checked.
Motivation:
parseInteger did not distinguish between not having enough bytes for an
integer and not being able to parse the integer that was present. This
was a bit tricky for code internally, where some call sites had extra
code looking for spooky action at a distance in order to determine if
the integer failed to parse.
This is unnecessary: parseInteger is sufficiently aware of what's going
on to address this problem itself.
Modifications:
- Added a new parser error (acceptable as we haven't tagged 1.0 yet).
- Throw it from parseInteger if the integer is invalid.
Result:
parseInteger clearly communicates if the integer failed to parse.
Motivation:
ByteBufferView is not zero indexed, but parseSimpleString assumes it is.
Modifications:
- Correctly compute on the distance between two indices.
- New, somewhat contrived, test case.
Result:
No functional change: because RediStack assumes the remote peer will
always correctly terminate with /r/n, there is no point at which this
code could misbehave in the current implementation. However, with small
changes it is possible to trigger it, as the new test demonstrates.
Motivation:
When we only want the first byte, rather than create temporary
intermediate arrays we can just ask NIO to give us the first byte. This
avoids unnecessary allocations.
Modifications:
- Replace `readBytes(length: 1).first` with `readInteger(as:
UInt8.self)`
Results:
11% performance improvement in load testing due to reduced allocator
pressure on the hot path.
Motivation:
When attempting to locate a single byte, creating a transient
ByteBufferView is an excessively heavyweight operation compared to a
simple getInteger. In particular, a BBV requires retain/release
operations to enforce the CoW invariants, as well as requires jumps
through substantial amounts of generic Collection code. While this can
be specialized, so can getInteger, and getInteger has much less code in
the way to cause costs.
Modifications:
- Replace temporary view creation with getInteger.
Results:
5% performance improvement on raw throughput tests.
Motivation:
`RESPValue` exposes a fair amount of complexity on how to intialize a single instance with the various overloads.
This aims to simplify the complexity for developers by providing a single initializer and relying on `RESPValueConvertible` to handle the complexities.
In addition, the Swift complier synthesizes a lot of default conformances that `RedisKey` has manually written, which is just unnecessary code.
Modifications:
- Rename: `RESPValue.init(_:)` to `RESPValue.init(from:)`
- Change: `RESPValue.init` `String?` and `FixedWidthInteger` overloads from `public` to `internal`
- Remove: Unnecessary code for various protocol conformances for `RedisKey`
Result:
Developers should have a direct and guided way of making instances of `RESPValue`
Motivation:
It was noticed that many of the commands are cumbersome to use with boilerplate type casting for each use that can be simplified within the library
by doing type conversion before returning the value to an end user.
Modifications:
Many APIs that return a `RESPValue` now have overloads to provide a `RESPValueConvertible` type that the value will be turned into before being returned.
For a few APIs that returned `RESPValue`, they did so as an Optional. Those APIs have been changed to always provide a `RESPValue` and return `.null` in cases where `nil` was returned.
In addition, the `@inlinable` attribute has been removed from any non-generic command API.
Result:
Developers should have less code boilerplate for turning values from `RESPValue` to their desired type with many commands.
Motivation:
The SortedSet and List range commands (LTRIM, LRANGE, ZRANGE, etc.) are stringly-based and not flexible with Swift syntax.
Modifications:
- Add overloads of LTRIM that support the gambit of Range Standard Library types
- Rework LRANGE to mirror LTRIM method signatures
- Rework ZScore Range based commands to be more type-safe with `RedisZScoreBound` enum
- Rework ZLex Range based commands to be more type-safe with `RedisZLexBound` enum
- Rework ZCOUNT, ZLEXCOUNT, ZRANGE, ZREVRANGE, ZREMRANGEBYLEX, ZREMRANGEBYRANK, ZREMRANGEBYSCORE methods to be more type-safe and support Swift Range syntax
Result:
Working with SortedSet ranges should be much more type safe, and expressive with Swift's Range syntax.
Motivation:
While reviewing the API, the current design does not read well, and still has room for misunderstanding the actual end result of a ZADD operation.
Modifications:
- Rename `RedisSortedSetAddOption` to `RedisZaddInsertBehavior` and update cases to match desired use site syntax.
- Add `RedisZaddReturnBehavior` enum to define how `zadd` should calculate the return value.
- Update `zadd` and its overloads to support the two new enums in the form of `zadd(_:to:inserting:returning:)`
Result:
The more "Swifty" API will make it much more clear to developers at the call site what the actual behavior of the ZADD command will be.
Motivation:
The goal is to have a strong-typed API for type-safety in arbitrary values, such as trying to use
Int to represent time - as '3' could mean any unit of time, leaving many places for errors and bugs.
Modifications:
Switch all current APIs that accept a `timeout` argument to use `NIO.TimeAmount` instead of a plain `Int`.
Result:
Developers will have an easier time reasoning about their own code as to what values might mean when working with
timeouts in Redis APIs.
Motivation:
It it pretty common as a developer when working with connections and "database" clients to want to know
if the connection is currently open before doing any work.
Modifications:
Add `var isConnected: Bool { get }` requirement to the `RedisClient` protocol
Result:
Developers should now have access to the connectivity state of any `RedisClient`
Motivation:
Inspired by Swift by Sundell's article on type-safe identifers, the goal of this commit is to have the compiler
assist in preventing incorrect Redis key values from being used in API calls.
See https://www.swiftbysundell.com/articles/type-safe-identifiers-in-swift/ for the inspiration.
Modifications:
- Add new `RedisKey` struct that wraps around a single `String` value that conforms to several expected protocols
(Hashable, Comparable, Codable, etc.)
- Change all command APIs to require `RedisKey` rather than plain strings
Result:
When encountering an API requiring a RedisKey, it should be much more apparant at the use site what form a value should take.
Motivation:
The library provides command implementations for almost every single Redis command, authorize is no different.
Modifications:
Add `authorize(with:)` command method on `RedisClient`
Replace the implementation in `RedisConnection.connect(...)`
Result:
Developers should now have independent access to the `AUTH` Redis command on `RedisClient` implementations.
Motivation:
Logging is more dynamic in real world usage than the current static heavy API allows.
Users generally want to be capable of updating connection logger metadata to attach dynamic properties such as an HTTP request ID for log tracing.
Modifications:
- Move all logs to `RedisConnection`
- Add `id: UUID` property to `RedisConnection`
- Add `logging` property and `setLogging(to:)` method requirements to `RedisClient`
- Add chainable `logging(to:)` method extension to `RedisClient`
- Add additional `trace` log statements to `RedisConnection`
- Change when `RedisConnection.init` logging and metric calls are made
- Change some `debug` log statements to `trace in `RedisConnection`
Result:
Users should have infinitely more flexibility in how RedisConnection, and RedisClient implementations in general, behave in regards to logging.
Motivation:
For ergonomics, users sometimes want to provide arguments as a variadic list rather than an array.
Modifications:
- Add variadic overloads for almost all methods that accept lists of homogenous types
Result:
Users should have more flexibility in the way arguments are passed to command methods
Motivation:
It is wrong to always assume that a GET operation is expecting a String response type, as users may be storing other types of data.
Modifications:
- Add `get` generic method with a constraint for types of `RESPValueConvertible` to convert values to the user desired type
- Change existing `get` method to specialize the generic overload
- Fix incorrect doc block regarding the ELF failure condition
Result:
Users should now be able to specialize the return type of a "GET" command
Motivation:
The assertions in `errorCaught(context:error:)` and `channelRead(context:data:)` are holdovers from when SwiftLog was not integrated into the package. The conditions where they were triggered are safely handled in other ways, so the asserts are not guarding against undefined behavior or bad app state.
Modifications:
Remove the two asserts regarding the `RedisCommandHandler.commandResponseQueue` to rely on Logging to bubble up errors to the user.
Result:
Debug builds using RediStack should not encounter unexpected assertions and should rely on other, better, error handling.
This contributes to issue #65
Motivation:
After enabling the thread sanitizer during testing, a data race with the `activeConnectionCount` in `RedisMetrics` was caught due to changing a primitive `Int` across threads.
Modifications:
Add a specialized class `ActiveConnectionGauge` to wrap the semantics of the `activeConnectionCount` with an `Atomic<Int>` property.
Result:
No data races should occur with the `RedisMetrics` subsystem.
Motivation:
While working to add more test coverage with `RESPTranslator`, it was made apparent that a `.bulkString(.none)` is impossible to create directly with the `RESPValue` initializers, even though it is a reasonable possibility.
Additionally, forcing all integer types to have to be stored in an `Int` is unnecessarily restrictive.
Modifications:
- Change `RESPValue.init(bulk:)` initializers to accept `Optional` instances
- Change `RESPValue.init(bulk:)` for `Int` initializer to be generic on `FixedWidthInteger`
Result:
Converting types to and from `RESPValue` should be more bi-directional and seamless.
Motivation:
Diagnostics for why `.bulkString` parses might fail were weak, and edge cases fell through gaps in coverage were found.
Modifications:
Added new cases to `RESPTranslator.ParsingError` for `.bulkString` parsing with additional test coverage.
Result:
Users should have better diagnostics for bogus data or failed parsing state.
Motivation:
After working with RedisKit with RediStackTestUtils as a dependency, it was realized how opinionated the module is in how RedisConnections can be created in test environments.
Modifications:
Require more information, with reasonable defaults for `RedisConnection.init()`. Provide subclass hooks for `RedisIntegrationTestCase` for implementors to make decisions for themselves at how to connect to Redis.
Result:
Users should have more freedom in how they connect to Redis in their units tests.
Motivation:
There is a reasonable way to compare if two `RedisErrors` are equal, which was seen as needed in the `Equatable` conformance for `RESPValue`.
Modifications:
Added `Equatable` conformance for `RedisError` by comparing the messages.
Result:
Two `RedisError` instances are now equatable.
Motivation:
While working on unit tests the need for conformance to `Equatable` for `RESPValue` has been needed a few times and it was decided to make it public.
Modifications:
Added conformance to `Equatable` for `RESPValue` with unit test.
Result:
Users should now be able to compare two `RESPValue` instances for equality.
Motivation:
During the refactor work (commit ea7c755) that was merged with MRs !71 and !53 - `sendCommandsImmediately` was accidentally lowered to `internal`
Modifications:
Properly mark `RedisConnection.sendCommandsImmediately` as `public`
Result:
Developers should now have proper access to the `sendCommandsImmediately` property
Motivation:
Issue #60 called for improving the type safety of the options available for the `zadd` command, and MR !70 made some great headway, but attempted to cram too much into a single enum.
Modifications:
- Break the `RedisSortedSetAddOption.returnChangedCount` value into an additional boolean param
Result:
Using `zadd` should now be more straight forward, while being type safe.
Motivation:
There are several cases where "command" could refer to a command keyword, or an entire message (keyword + args). This made working with `RedisCommand` and it's documentation ambiguous.
Modifications:
- Rename `RedisCommand.command` to `message`
- Rename initializer labels to `message` and `responsePromise`
Result:
When encountering a `RedisCommand` everyone should know that they are dealing with a message that should be sent to Redis as soon as possible.
Motivation:
To make it a little more generic, and to avoid turnover during renames (such as the planned rebranding in issue #61), `RedisClientError` more accurately reflects the source of the errors, as well as the responsibility of causing the bug.
Modifications:
- Rename `RedisNIOError` to `RedisClientError`
- Rename `RedisError` file to `RedisErrors`
- Add documentation of `RedisClientError`
- Remove no longer used `.unsupportedOperation(method:message:)` value
- Rename `.responseConversion(to:)` to `.failedRESPConversion(to:)`
Result:
Names of `RedisClientError` should be more descriptive, less prone to turnover, and more documented for users to understand the issues related to these thrown errors.
Motivation:
During proposal review, and while working within the codebase, several issues were identified with how `RedisConnection` was architectured.
Modifications:
- Change implementation of `RedisConnection` in some areas for new logic of internal `ConnectionState`
- Change behavior of logging in a few places
- The initializer for `RedisConnection` is now **internal**
- How users can override the default `ClientBootstrap` for a connection is by passing an instance to the `.connect` static method
- Change unit tests to inherit from a common XCTestCase class that handles creation and cleanup of `RedisConnection` in tests
- Remove Redis namespace enum
Result:
The API for `RedisConnection` should be much simpler, with the implementation being less buggy.
This resolves issues #49, #54, and #57.
Motivation:
While working through issue #59, it was noticed just how "stringly" the SortedSet command options for `zadd`, `zinterstore`, and `zunionstore` were, and Swift provides ways of having strong type safety for these options.
Modifications:
- Add `RedisSortedSetAddOption` and `RedisSortedSetAggregateMethod` to replace the String API in `zadd`, `zinterstore`, and `zunionstore`
- Fix an implication of how `overestimatedCountBeingAdded` documentation for `Array where Element == RESPValue` for `add(contentsOf:overestimatedCountBeingAdded:_:)`
Result:
Users should have a more discoverable and straightforward way that isn't error prone for calling `zadd`, `zinterstore`, and `zunionstore` with Redis supported options.
Motivation:
Johannes continues to provide great insight, and correctly pointed out that `RESPValueConvertible` was being used as an "existential" in all cases.
This can cause unexpected type-erasure and introduce unnecessary cost overhead with dynamic dispatch when in most cases we know the exact value we want for `RESPValue` to execute commands.
Modifications:
- Add new extensions to `Array where Element == RESPValue` for appending and adding elements into them
- Change `RedisClient.send(command:with:)` to require `[RESPValue]` instead of `[RESPValueConvertible]` as the `with` argument type
- Change all instances of `RESPValueConvertible` being an "existential" type for method arguments to instead be a generic constraint
Result:
The library should be safeguarded from a class of bugs, with the use of `send` being a bit more straight forward, with some new convenience methods for `[RESPValue]` types.
Motivation:
Johannes provided a fair code review of the project and summarized his findings in issue #48, and one of the prime offenders was all of the `unsafe*` APIs (pointers, buffers, bytes)
that were used with `RESPValue` and `RESPValueConvertible`.
He also provided great feedback and pointed out good points of confusion with the API design of `RESPValue` and `RESPValueConvertible`.
Modifications:
- Return to using `Array` instead of `ContiguousArray` for `RESPValue.array` storage
- Update all documentation to be more thorough in explaining how the types should be used and conformed to.
- Remove all uses of `unsafe*` APIs where possible
- Change implementations to be a lot more type and memory safe, double checking assumptions
- Remove conformance to `ExpressibleBy*Literal` as it is too easy for users to shoot themselves in the foot and saves only a few characters over `.init(bulk:)`
- Create new `RedisNIOTestUtils` target for common test extensions, making them public
- Move most almost all implementations of `RESPValue` computed properties into the `RESPValueConvertible` conformances
Result:
Users should be more safeguarded by the API against unknowingly getting incorrect `RESPValue` representations, the API design of `RESPValue` and `RESPValueConvertible` should be much clearer,
and memory safety should be at a higher bar from these changes.
This resolves issues #55 & #48, and contributes to issue #47.
Motivation:
During proposal review, it was noted that `RedisCommandContext` was a bit misleading, and the hidden reference semantics worrisome.
In addition, several of parts of the documentation around `RedisCommandHandler` were weak or also misleading.
Modifications:
- Rename `RedisCommandContext` to just `RedisCommand`
- Update documentation to be more explicit about the module who owns the types being referenced
- Update documentation to call out explicit usage semantics and behavior
- Change `RedisCommandHandler` to close the socket connection on error thrown
- Rename the "base" Redis Channel Handlers to be more explicitly named
Result:
Users should have clearer documentation on what happens when using `RedisCommandHandler` and `RedisCommand` without hidden semantics.
This contributes to issue #47.
Motivation:
During proposal review, it was appropriately pointed out that `RedisClient.expire` incorrectly mixes 'deadline' and 'timeout' terminology.
Modifications:
- Change references of 'deadline' to 'timeout' to follow Redis' established semantics for 'EXPIRE'
- Add additional unit test for `RedisClient.expire`
Result:
`RedisClient.expire` should now be more clear as to its semantics and not mix terminology incorrectly.
This contributes to issue #47
Motivation:
During proposal review, feedback was provided that the discoverability of the factory method for building a standard RESP `ChannelPipeline` was poor outside of documentation.
Modifications:
- Move `Redis.makeDefaultClientBootstrap` to `ClientBootstrap.makeRedisTCPClient`.
- Move the `channelInitializer` implementation into a new `Channel.addBaseRedisHandlers()` instance method.
Result:
Users should have an easier time discovering how to easily create baseline RESP `ChannelPipelines`.
This contributes to #47.
Motivation:
During proposal review, it was pointed out that the code with a position index was redundant and error prone over relying on `ByteBuffer`'s `readerIndex`.
Modifications:
Refactored `RESPTranslator` to rely on `ByteBuffer.readerIndex` for position of current parsing cursor,
and eliminated `ParsingResult` enum to instead return `RESPValue?`.
The implementation for writing out `RESPValue` has been expanded to `RESPValueConvertible` and moved to an extension of `ByteBuffer`.
Result:
Parsing `RESPValue` into and out of `ByteBuffer` should be less error-prone and more straight forward.
This contributes to issues #47 and #55
Motivation:
During SSWG review, feedback was provided on the API design of the `RESPValueConvertible.init` signature and how it should appropriately follow Swift Design Guidelines regarding labels.
Modifications:
`RESPValueConvertible.init(_:` is now `RESPValueConvertible.init(fromRESP:)`.
Result:
There should be more clarity at the call site when initializing a type from a `RESPValue`.
This contributes to #47
Motivation:
During SSWG review, feedback was provided that forced a re-evaluation of early design feedback interpretation on composability of RedisNIO.
First understanding was that the desire was to have "customization points" to gain benefits of implementation of RedisCommandHandler, while new understanding is that it just needs to be public in order for users to include it in their own custom ChannelPipeline schemes.
Modifications:
`RedisCommandHandler` is now a final class.
Result:
Users will no longer be able to subclass `RedisCommandHandler`, but gain a super slight performance increase.
This contributes to #47.
Nathan Harris
George Barnett
Cory Benfield
Nathan Harris
Ondrej Rafaj