tarpc is now instrumented with tracing primitives extended with
OpenTelemetry traces. Using a compatible tracing-opentelemetry
subscriber like Jaeger, each RPC can be traced through the client,
server, amd other dependencies downstream of the server. Even for
applications not connected to a distributed tracing collector, the
instrumentation can also be ingested by regular loggers like env_logger.
# Breaking Changes
## Logging
Logged events are now structured using tracing. For applications using a
logger and not a tracing subscriber, these logs may look different or
contain information in a less consumable manner. The easiest solution is
to add a tracing subscriber that logs to stdout, such as
tracing_subscriber::fmt.
## Context
- Context no longer has parent_span, which was actually never needed,
because the context sent in an RPC is inherently the parent context.
For purposes of distributed tracing, the client side of the RPC has all
necessary information to link the span to its parent; the server side
need do nothing more than export the (trace ID, span ID) tuple.
- Context has a new field, SamplingDecision, which has two variants,
Sampled and Unsampled. This field can be used by downstream systems to
determine whether a trace needs to be exported. If the parent span is
sampled, the expectation is that all child spans be exported, as well;
to do otherwise could result in lossy traces being exported. Note that
if an Openetelemetry tracing subscriber is not installed, the fallback
context will still be used, but the Context's sampling decision will
always be inherited by the parent Context's sampling decision.
- Context::scope has been removed. Context propagation is now done via
tracing's task-local spans. Spans can be propagated across tasks via
Span::in_scope. When a service receives a request, it attaches an
Opentelemetry context to the local Span created before request handling,
and this context contains the request deadline. This span-local deadline
is retrieved by Context::current, but it cannot be modified so that
future Context::current calls contain a different deadline. However, the
deadline in the context passed into an RPC call will override it, so
users can retrieve the current context and then modify the deadline
field, as has been historically possible.
- Context propgation precedence changes: when an RPC is initiated, the
current Span's Opentelemetry context takes precedence over the trace
context passed into the RPC method. If there is no current Span, then
the trace context argument is used as it has been historically. Note
that Opentelemetry context propagation requires an Opentelemetry
tracing subscriber to be installed.
## Server
- The server::Channel trait now has an additional required associated
type and method which returns the underlying transport. This makes it
more ergonomic for users to retrieve transport-specific information,
like IP Address. BaseChannel implements Channel::transport by returning
the underlying transport, and channel decorators like Throttler just
delegate to the Channel::transport method of the wrapped channel.
# References
[1] https://github.com/tokio-rs/tracing
[2] https://opentelemetry.io
[3] https://github.com/open-telemetry/opentelemetry-rust/tree/main/opentelemetry-jaeger
[4] https://github.com/env-logger-rs/env_logger
# New Crates
- crate rpc contains the core client/server request-response framework, as well as a transport trait.
- crate bincode-transport implements a transport that works almost exactly as tarpc works today (not to say it's wire-compatible).
- crate trace has some foundational types for tracing. This isn't really fleshed out yet, but it's useful for in-process log tracing, at least.
All crates are now at the top level. e.g. tarpc-plugins is now tarpc/plugins rather than tarpc/src/plugins. tarpc itself is now a *very* small code surface, as most functionality has been moved into the other more granular crates.
# New Features
- deadlines: all requests specify a deadline, and a server will stop processing a response when past its deadline.
- client cancellation propagation: when a client drops a request, the client sends a message to the server informing it to cancel its response. This means cancellations can propagate across multiple server hops.
- trace context stuff as mentioned above
- more server configuration for total connection limits, per-connection request limits, etc.
# Removals
- no more shutdown handle. I left it out for now because of time and not being sure what the right solution is.
- all async now, no blocking stub or server interface. This helps with maintainability, and async/await makes async code much more usable. The service trait is thusly renamed Service, and the client is renamed Client.
- no built-in transport. Tarpc is now transport agnostic (see bincode-transport for transitioning existing uses).
- going along with the previous bullet, no preferred transport means no TLS support at this time. We could make a tls transport or make bincode-transport compatible with TLS.
- a lot of examples were removed because I couldn't keep up with maintaining all of them. Hopefully the ones I kept are still illustrative.
- no more plugins!
# Open Questions
1. Should client.send() return `Future<Response>` or `Future<Future<Response>>`? The former appears more ergonomic but it doesn’t allow concurrent requests with a single client handle. The latter is less ergonomic but yields back control of the client once it’s successfully sent out the request. Should we offer fns for both?
2. Should rpc service! Fns take &mut self or &self or self? The service needs to impl Clone anyway, technically we only need to clone it once per connection, and then leave it up to the user to decide if they want to clone it per RPC. In practice, everyone doing nontrivial stuff will need to clone it per RPC, I think.
3. Do the request/response structs look ok?
4. Is supporting server shutdown/lameduck important?
Fixes#178#155#124#104#83#38
* Head off imminent breakage due to https://github.com/rust-lang/rust/pull/51285.
* Fix examples and documentation to use a recently-gated feature, `proc_macro_path_invoc`.
* Update dependency versions.
* Create a directory for the `future::server` module, which has become quite large. server.rs => server/mod.rs. Server submodules for shutdown and connection logic are added.
* Add fn thread_pool(...) to sync::server::Options
* Configure idle threads to expire after one minute
* Add tarpc::util::lazy for lazily executing functions. Similar to `futures::lazy` but useful in different circumstances. Specifically, `futures::lazy` typically requires a closure, whereas `util::lazy` kind of deconstructs a closure into its function and args.
* Remove some unstable features, and `cfg(plugin)` only in tests. Features `unboxed_closures` and `fn_traits` are removed by replacing manual Fn impls with Stream impls. This actually leads to slightly more performant code, as well, because some `Rc`s could be removed.
* Fix tokio deprecation warnings. Update to use tokio-io in lieu of deprecated tokio-core items. impl AsyncRead's optional `unsafe fn prepare_uninitialized_buffer` for huge perf wins
* Add debug impls to all public items and add `deny(missing_debug_implementations)` to the crate.
* Bump tokio core version.
Unfortunately, cargo's semantic versioning gets confused by the
"-alpha" suffix in current bincode versions (I think): even though
tarpc's Cargo.toml specified version "1.0.0-alpha4", cargo will
download the more recent "1.0.0-alpha6", which has breaking changes
to the `SizeLimit` enum.
This change makes tarpc work with bincode 1.0.0-alpha6 and updates the
dependency.
* Change `client::{future, sync}, server::{future, sync}` to `future::{client, server}, sync::{client, server}`
This reflects the most common usage pattern and allows for some types to not need to be fully qualified when used together (e.g. the previously-named `client::future::Options` and `server::future::Options` can now be `client::Options` and `server::Options`).
* sync::client: create a RequestHandler struct to encapsulate logic of processing client requests.
The largest benefit is that unit testing becomes easier, e.g. testing that the request processing stops when all request senders are dropped.
* Rename Serialize error variants to make sense.
* Rename tarpc_service_ConnectFuture__ => Connect (because it's public)
* Use tokio proto's ClientProxy.
Rather than reimplement the same logic. Curiously, this commit
isn't a net loss in LOC. Oh well.
* Factor out os-specific functionality in listener() into their own fns
* Remove service-fn dep
* feat(serde): upgrade serde to 0.9
If you are using `#[derive(Serialize, Deserialize)]` or implementing
your own (de)serialization behavior for your RPC types you will need to
ensure you are using serde 0.9.x.
[breaking-change]
* chore(byteorder): upgrade byteorder to 1.0
When `-- features tls` is specified for tarpc, RPC communication can
also occur over a `TlsStream<TcpStream>` instead of a `TcpStream`.
* The functional tests have been refactored to use a common set of
functions for constructing the client and server structs so that all
the tests are shared across non-tls and tls test runs.
* Update pre-push to test TLS
* The `cfg_attr` logic caused many false warnings from clippy, so for now the crate docs for TLS are not tested.
* Extend snake_to_camel plugin to replace {} in the doc string with the origin snake-cased ident.
Also, track tokio-rs master.
This is really ad-hoc, undiscoverable, and unintuitive, but there's no way to programmatically create doc strings
in regular code, and I want to produce better doc strings for the associated types.
Given `fn foo_bar`:
Before: `/// The type of future returned by the function of the same name.`
After: ``/// The type of future returned by `{}`.``
=> `/// The type of future returned by foo_bar.`
* Fix some docs
* Use a helper fn on pipeline::Frame instead of handrolled match.
* Don't hide docs for ClientFuture.
It's exposed in the Connect impl of FutureService -- the tradeoff for not generating *another* item -- and hiding it breaks doc links.
* Formatting
* Rename snake_to_camel plugin => tarpc-plugins
* Update README
* Mangle a lot of names in macro expansion.
To lower the chance of any issues, prefix idents in service expansion with __tarpc_service.
In future_enum, prefix with __future_enum. The pattern is basically __macro_name_ident.
Any imported enum variant will conflict with a let binding or a function arg, so we basically
can't use any generic idents at all. Example:
enum Req { request(..) }
use self::Req::request;
fn make_request(request: Request) { ... }
^^^^^^^ conflict here
Additionally, suffix generated associated types with Fut to avoid conflicts with camelcased rpcs.
Why someone would do that, I don't know, but we shouldn't allow that wart.
* Rewrite tarpc on top of tokio.
* Add examples
* Move error types to their own module.
Also, cull unused error variants.
* Remove unused fn
* Remove CanonicalRpcError* types. They're 100% useless.
* Track tokio master (WIP)
* The great error revamp.
Removed the canonical rpc error type. Instead, the user declares
the error type for each rpc:
In the above example, the error type is Baz. Declaring an error is
optional; if none is specified, it defaults to Never, a convenience
struct that wraps the never type (exclamation mark) to impl Serialize, Deserialize,
Error, etc. Also adds the convenience type StringError for easily
using a String as an error type.
* Add missing license header
* Minor cleanup
* Rename StringError => Message
* Create a sync::Connect trait.
Along with this, the existing Connect trait moves to future::Connect. The future
and sync modules are reexported from the crate root.
Additionally, the utility errors Never and Message are no longer reexported from
the crate root.
* Update readme
* Track tokio/futures master. Add a Spawn utility trait to replace the removed forget.
* Fix pre-push hook
* Add doc comment to SyncServiceExt.
* Fix up some documentation
* Track tokio-proto master
* Don't set tcp nodelay
* Make future::Connect take an associated type for the future.
* Unbox FutureClient::connect return type
* Use type alias instead of newtype struct for ClientFuture
* Fix benches/latency.rs
* Write a plugin to convert lower_snake_case idents/types to UpperCamelCase.
Use it to add associated types to FutureService instead of boxing the return futures.
* Specify plugin = true in snake_to_camel/Cargo.toml. Weird things happen otherwise.
* Add clippy.toml
