Currently there is no way to set `RedisConnection`'s `onUnexpectedClosure` for connections within a `RedisConnectionPool`. This PR adds a new closure to `RedisConnectionPool.onUnexpectedConnectionClose` which will be triggered for every pool connection that closed unexpectedly.
## Motivation
Since Redis 6.0, a new serialization protocol format (v3) is available that gives richer semantic reasoning behind the different types to enable commands to better understand the return types to provide in their programming language.
In addition, the `RESPTranslator` type is going to see more direct usage, and the current API doesn't make read well.
## Changes
- Add: Internal `RESPVersion` enum that the `RESPTranslator` will start to use
- Rename: `RESPTranslator.parseBytes` to `RESPTranslator.read(from:)`
Motivation
Right now the PubSub handlers are split into three separate closures, with the subscribe/unsubscribe handlers being optional. This won't play well with AsyncStream for being able to respond to all events that a PubSub subscription can cause.
Additionally, the current structure is very verbose in code to maintain - but also adds complexity to developers who are first getting started to understand the lifecycle of PubSub events.
Changes
- Add: New `RedisPubSubEvent` enum that captures the subscribe, unsubscribe, and message lifecycle events
- Add: New `RedisPubSubEventReceiver` that combines the previous 3 closure types
- Add: Dedicated DocC Symbol Extension file for `RedisPubSubHandler`
- Change: `RedisClient.subscribe` and `RedisClient.psubscribe` method signatures to only require a single unlabeled closure
- Rename: `RedisUnsubscribeEventSource` to `RedisPubSubEvent.UnsubscribeEventSource`
- Remove: `RedisSubscriptionMessageReceiver`, `RedisSubscriptionChangeDetails`, `RedisSubscribeHandler`, and `RedisUnsubscribeHandler` types
Result
Developers should have a much easier time getting started and understanding PubSub with assistance from the compiler with types to understand
what they're being given and what's available to them as information to make more informed decisions in their app logic.
## Motivation
The API for establishing the configuration of a connection pool had a lot of jargon and properties that developers had issues keeping straight and understanding what each does.
This commit provides first-class API support for concepts such as retry strategies, and how the pool handles connection counts.
## Changes
- Add: New ConnectionCountBehavior for determining leaky / non-leaky behavior
- Add: New ConnectionRetryStrategy for allowing customization of retry behavior
- Change: RedisConnection.defaultPort to be a computed property
- Change: The logging keys of pool connection retry metadata
- Rename: Several configuration properties to drop prefixes or to be combined into new structures
## Result
Developers should have a much better experience exploring the available configuration options for pools and connections, being able to understand how each piece works with the underlying system.
There are many times that developers want exact control over which EventLoop will be executing their chained EventLoopFuture callbacks
and which Logger will do the logging in calls deep within RediStack.
All commands will now accept an optional EventLoop and Logger to hop to, and using the logger for desired logs.
Motivation:
Following the SSWG guidelines for libraries and log levels, because much of the library's behavior is expressed in the language and NIO framework as errors and failed ELFs, logging at error is "verbose" and takes away control from developers.
Modifications:
Log messages have been adjusted to more accurately represent when and how the log message should be used, especially when ELFs are failed or errors are thrown.
Result:
Developers won't have log messages at error or critical unless they opt-in from their own code, unless the library has no way of expressing the failure condition through the language.
The newly-released Service Discovery framework gives us the interesting
opportunity to make RediStack aware of complex service discovery tools.
This patch supplies a simple adaptor to integrat Service Discovery with
RediStack's pooled client, allowing users to work with arbitrary service
discovery systems.
In some circumstances users may have connection pools configured without
any SocketAddresses ready to go. This is particularly likely in service
discovery configurations. Right now, the effect of attempting to use
such a pool is two fold. First, we'll emit a bunch of error level logs
telling users we have no addresses. Second, we'll fall into the
exponential backoff phase of connection establishment.
The first property is annoying, but the second one is actively harmful.
If your construction is timed incorrectly, we'll have the awkward
problem of burning a bunch of CPU trying to create connections we know
we cannot, and then a lengthy delay after the addresses are actually
configured before we start trying to use them. That's the worst of all
worlds.
This patch adds logic to detect the attempt to create connections when
we don't have any configured addresses and delays them. This path should
improve performance and ergonomics when in this mode.
Motivation:
RediStack today represents a command as a temporary object for the purpose of writing to the channel.
While it is useful to have an object for that purpose, commands handled in this way require immediate execution
and aren't available for other purposes.
Commands can serve a better purpose as a lightweight object to support delayed execution,
so that pipeling as described in issue #63 could be possible.
Modifications:
- Add: `get` overloads for JSON codable interaction on `RedisClient`
- Add: New `RedisZRangeResultOption` type for better interactions with zrange operations that can optionally return scores
- Add: New `RedisHashFieldKey` for type-safety when working with Hash field keys much like `RedisKey`
- Change: A few API types from enums to structs for library evolution
- Change: `RedisCommandHandler` to operate on a tuple of `RESPValue, EventLoopPromise<RESPValue>` rather than `RedisCommand`
- Change: `RedisCommand` to be a generic struct with the keyword, arguments, and a transform closure to defer execution
- Change: Almost all `RedisClient` command extensions to be factory methods on `RedisCommand` instead
- Change: Many response types to be optional to avoid developers having to do `isNull` checks on their own
- Change: `RedisClient.send(command:arguments:)` to be generic with `send(_:)` as the signature
- Rename: RedisClient extensions for scan methods to be more discoverable and legible as `scanHashField`, etc.
Result:
It should be easier to support a clean pipelining API with deferred command execution,
with extensions being easier for 2nd party developers, and the maintenance overhead of all of the command extensions
should be a little lighter when it comes to changing HOW commands are sent such as adding a context parameter
Motivation:
`RedisKeyLifetime` already has "RedisKey" as a prefix so it naturally fits as a nested type.
Modifications:
- Change: `RedisKeyLifetime` to be nested in `RedisKey` and named `Lifetime`
- Rename: `RedisKeyLifetime.Lifetime` to `Duration`
- Deprecate: `RedisKeyLifetime` and the nested type `Lifetime`
Result:
The global namespace is a little less cluttered with the types falling naturally where they already are.
Motivation:
When `RedisConnection.allowSubscriptions` is set to `false`, the connection could still be in a subscription state
leaving further commands to fail slowly from a full roundtrip to Redis, rather than succeeding as expected.
This changes the implementation so that it triggers a full unsubscribe from patterns and channels when set to `false`.
Modifications:
- Change: `RedisConnection.allowSubscriptions` to call `unsubscribe()` and `punsubscribe()` when set to `false`
- Change: `RedisPubSubHandler` to prefix storage of all dictionary keys to avoid name clashes between pattern and channel subscriptions
Result:
Developers should now have more deterministic and unsurprising behavior with PubSub
in regards to subscription management and connection state.
Motivation:
The methods of unsubscribing from all channels / patterns were not working as expected as they need to be special-case handled.
Modifications:
- Change: `RedisPubSubHandler` to be special-case unsubscribe when no arguments are provided
Result:
Developers should now properly be able to unsubscribe from all channels / patterns with a single method call.
Motivation:
Some Redis commands are very connection specific that have impacts on future access that makes it difficult in the current
checkout-use-return cycle that `RedisConnectionPool` uses.
Developers need a way to borrow a specific connection, chain several commands together, and then return the connection to the pool.
Modifications:
- Add: `leaseConnection` method to `RedisConnectionPool` which provides a connection from the pool and returns it after a provided closure's ELF resolves
- Add: `allowSubscriptions` property to `RedisConnection` for controlling the ability to make PubSub subscriptions
- Add: `RedisClientError.pubsubNotAllowed` case for when `RedisConnection.allowSubscriptions` is set to `false` and a subscription was still attempted
Result:
Developers should now have an "escape hatch" with `RedisConnectionPool` to do limited exclusive chains of operations on a specific connection.
Motivation:
When trying to allow users to configure the connection retry timeout offset,
not having a value provided (deadline of `now`) caused all attempts to use the pool to fail.
Modifications:
- Change: RedisConnectionPool to always have a timeout offset defined
Result:
If users don't specify any value, then the default of 60 seconds will be used.
If users specify "nil" (or `.none`) as the timeout, then a minimum of 10 milliseconds will be used to avoid immediate timeouts
Otherwise, use the user's specified `TimeAmount` as the offset of the timeout
Motivation:
With RedisConnectionPool a timeout is provided to prevent infinite loops of
retrying connections, but right now it is hardcoded to 60 seconds.
Users of downstream projects such as Vapor are noticing a "regression" of sorts, as previously
EventLoopFutures would fail immediately if a connection was not made available.
Modifications:
- Add: `connectionRetryTimeout` parameter to `RedisConnectionPool` initializer that still defaults to 60 seconds
- Change: RedisConnectionPool to use the new parameter if available to offset a deadline from "now"
Result:
Users can now configure the connection pool to fail immediately if connections are not available.
Motivation:
To ship PubSub faster, it was merged to the master branch without a peer review. This commit is to address the critical points of feedback given in a post-commit review.
Modifications:
- Add: New RedisClientError case where a "race condition" of removing a pubsub handler and subscription can happen
- Add: New state to RedisPubSubHandler for when it has been removed from a ChannelPipeline
- Change: RedisPubSubHandler to require an `eventLoop` in its initializer
- Change: The subscribe and unsubscribe methods on RedisPubSubHandler to handle the EventLoop hopping to be thread-safe
Result:
PubSub should have a more robust and thread-safe implementation.
Motivation:
One of the great features of Redis is being able to subscribe and receive messages published to specific channels
as a way of acting as a message queue for processing jobs.
PubSub requires a specific understanding of the connection model that can only be implemented directly in this library.
Modifications:
- Add: `RedisPubSubHandler` to sit in front of `RedisCommandHandler` to manage subscription callbacks and Redis registration
- Add: `publish` and the `pubsub` commands
- Add: `addPubSubHandler` extension to `NIO.Channel`
- Add: Type-safe String wrapper of `RedisChannelName` for PubSub methods
- Add: `pubsubSubscriptionNotFound` error case
- Add: `isSubscribed` property to `RedisConnection`
- Add: `availableConnectionCount` and `leasedConnectionCount` properties to `RedisConnectionPool`
- Add: Metrics for PubSub
- Add: `makeNewPool` factory method to `RedisConnectionPoolIntegrationTestCase`
- Change: `RedisClient` to require methods for PubSub management, as they are intrinsicly tied to the client's connection model
- Change: Parsing of `PING` response for handling special case in PubSub mode
- Rename: `ActiveConnectionGauge` to `RedisMetrics.IncrementalGauge`
Result:
Developers will now be able to use Redis in PubSub mode with both connections and pools.
This resolves#6
Marius Seufzer
Gwynne Raskind
Nathan Harris
Fabian Fett
Nathan Harris
Michael Stegeman
Daniel Ramteke
Peter Adams
Cory Benfield