This branch modifies the `tokio_trace::Span` API functions that take
span IDs (the `Span::child_of` constructor, and the `Span::follows_from`
method) so that more types bearing a span ID can be passed as an
argument. Span IDs may now be passed directly without requiring them to
be passed as `Some(id)`. This should make the API slightly more
ergonomic.
Also, it changes the `Span::field` method to take an `AsField` rather
than a `Borrow<str>`.
Signed-off-by: Eliza Weisman <[email protected]>
This branch improves the `fmt::Debug` implementation for `Metadata`,
and adds `fmt::Display` implementations for `FieldSet` and `ValueSet`.
When formatting a `Metadata`, only present fields are formatted --- if
optional fields, such as the file, line number, and module path are
`None`, they will be excluded. In addition, `Metadata` now formats its
`FieldSet` using `FieldSet`'s `fmt::Display` implementation, which is a
bit less noisy. Finally, the `Debug` output for `Metadata` now includes
the callsite that the metadata originates from.
The intention behind these changes is to make the output from failed
tests somewhat easier to interpret.
Signed-off-by: Eliza Weisman <[email protected]>
## Motivation
`tokio-trace` currently offers a strategy for compatibility with the
`log` crate: its macros can be dropped in as a replacement for `log`'s
macros, and a subscriber can be used that translates trace events to log
records. However, this requires the application to be aware of
`tokio-trace` and manually set up this subscriber.
Many libraries currently emit `log` records, and would like to be able
to emit `tokio-trace` instrumentation instead. The `tokio` runtimes are
one such example. However, with the current log compatibility strategy,
replacing existing logging with trace instrumentation would break
`tokio`'s logs for any downstream user which is using only `log` and not
`tokio-trace`. It is desirable for libraries to have the option to emit
both `log` _and_ `tokio-trace` diagnostics from the same instrumentation
points.
## Solution
This branch adds a `log` feature flag to the `tokio-trace` crate, which
when set, causes `tokio-trace` instrumentation to emit log records as well
as `tokio-trace` instrumentation.
## Notes
In order to allow spans to log their names when they are entered and
exited even when the span is disabled, this branch adds an
`&'static Metadata` to the `Span` type. This was previously stored in
the `Inner` type and was thus only present when the span was enabled.
This makes disabled spans one word longer, but enabled spans remain
the same size.
Fixes: #949
Signed-off-by: Eliza Weisman <[email protected]>
## Motivation
`tokio-trace` should have static verbosity level filtering, like the
`log` crate. The static max verbosity level should be controlled at
compile time with a set of features. It should be possible to set a
separate max level for release and debug mode builds.
## Solution
We can do this fairly similarly to how the `log` crate does it:
`tokio-trace` should export a constant whose value is set based on the
static max level feature flags. Then, we add an if statement to the
`span!` and `event!` macros which tests if that event or span's level
is enabled.
Closes#959
## Motivation
In order to implement "out of band" `Subscriber` APIs in third-party
subscriber implementations (see [this comment]) users may want to
downcast the current `Dispatch` to a concrete subscriber type.
For example, in a library for integrating `tokio-trace` with a fancy new
(hypothetical) distributed tracing technology "ElizaTracing", which uses
256-bit span IDs, we might expect to see a function like this:
```rust
pub fn correlate(tt: tokio_trace::span::Id, et: elizatracing::SpanId) {
tokio_trace::dispatcher::with(|c| {
if let Some(s) = c.downcast_ref::<elizatracing::Subscriber>() {
s.do_elizatracing_correlation_magic(tt, et);
}
});
}
```
This allows users to correlate `tokio-trace` IDs with IDs in the
distributed tracing system without having to pass a special handle to
the subscriber through application code (as one is already present in
thread-local storage, but with its type erased).
## Solution
This branch makes the following changes:
* Add an object-safe `downcast_raw` method to the `Subscriber` trait,
taking a `TypeId` and returning an `*const ()` if the type ID
matches the subscriber's type ID, or `None` if it does not, and
* Add `is<T>` and `downcast_ref<T>` functions to `Subscriber`
and `Dispatch`, using `downcast_raw`.
Unlike the approach implemented in #950, the `downcast_raw` method is
object-safe, since it takes a `TypeId` rather than a type _parameter_
and returns a void pointer rather than an `&T`. This means that
`Subscriber` implementations can override this method if necessary. For
example, a `Subscriber` that fans out to multiple component subscribers
can downcast to their component parts, and "chained" or "middleware"
subscribers, which wrap an inner `Subscriber` and modify its behaviour
somehow, can downcast to the inner type if they choose to.
[this comment]: https://github.com/tokio-rs/tokio/issues/932#issuecomment-469473501
[`std::error::Error`'s]: https://doc.rust-lang.org/1.33.0/src/std/error.rs.html#204
Refs: #950, #953, https://github.com/tokio-rs/tokio/issues/948#issuecomment-469444293
Signed-off-by: Eliza Weisman <[email protected]>
This branch makes the following changes to `tokio-trace`'s `Span` type:
* **Remove manual close API from spans**
In practice, there wasn't really a use-case for this, and it
complicates the implementation a bit. We can always add it back later.
* **Remove generic lifetime from `Span`**
Again, there wasn't actually a use-case for spans with metadata that
doesn't live for the static lifetime, and it made using `Span`s in
other types somewhat inconvenient. It's also possible to implement an
alternative API for non-static spans on top of the `tokio-trace-core`
primitives.
Signed-off-by: Eliza Weisman <[email protected]>
PR #973 changed the `tokio_trace_core::span::Id::from_u64` function to
require that the provided `u64` be greater than zero. However, I had
forgotten that the implementation of `Subscriber` for the `NoSubscriber`
type (which is used when no default subscriber is set) always returned
`span::Id::from_u64(0)` from its `new_span` method. In combination with
the assert added in #973, this means that every time a span is hit when
no subscriber is set, `tokio-trace-core` will panic.
This branch fixes the panics by having `NoSubscriber` construct span IDs
using a different (arbitrarily chosen) non-zero constant.
Signed-off-by: Eliza Weisman <[email protected]>
This branch changes `tokio_trace_core::span::Id::from_u64` to assert
that the integer from which the span ID is constructed is greater than
zero. This is to enable future use of non-zero optimization.
Unfortunately, we can't actually use a `NonZeroU64` _now_, as that type
was only stabilized in Rust 1.28.0, and `tokio`'s current minimum
supported Rust version is 1.26.0.
Adding and documenting the assertion now allows us to change the
internal representation to `NonZeroU64` later (when 1.28.0 is the
minimum supported Rust version), without causing a breaking change.
Signed-off-by: Eliza Weisman <[email protected]>
* trace-core: Pass dispatcher by ref to `dispatcher::with_default`
As requested by @carllerche in https://github.com/tokio-rs/tokio/pull/966#discussion_r264380005, this branch changes the
`dispatcher::with_default` function in `tokio-trace-core` to take the
dispatcher by ref and perform the clone internally. This makes this
function more consistant with other `with_default` functions in other
crates.
Signed-off-by: Eliza Weisman <[email protected]>
* trace: Don't set the default dispatcher on entering a span
Setting the default dispatcher on span entry is a relic of when spans
tracked their parent's ID. At that time, it was necessary to ensure that
any spans created inside a span were observed by the same subscriber
that originally provided the entered span with an ID, as otherwise, new
spans would be created with parent IDs that did not originate from that
subscriber.
Now that spans don't track their parent ID, this is no longer necessary.
However, removing this behavior does mean that if a span is entered
outside of the subscriber context it was created in, any subsequent
spans will be observed by the current default subscriber and thus will
not be part of the original span's trace tree. Since subscribers are not
expected to change frequently, and spans are not expected to move
between them, this is likely acceptable.
I've removed the tests for the old behavior.
Note that this change improves the performance of span entry/exit fairly
significantly. Here are the results of running a benchmark that enters
a span, does nothing, and immediately exits it, before this change:
```
test enter_span ... bench: 93 ns/iter (+/- 14)
```
...and after:
```
test enter_span ... bench: 51 ns/iter (+/- 9)
```
Signed-off-by: Eliza Weisman <[email protected]>
## Motivation
Currently, it isn't possible to import individual macros from
`tokio-trace` using the macros 1.2 syntax:
```rust
use tokio_trace::{debug, info, span};
```
This is because these macros require that `callsite` and `enabled` are
imported as well.
## Solution
This branch resolves the problem by adding the [`local_inner_macros`]
attribute to the instrumentation API's macros. This allows other macros
from within the crate to be used without requiring them to be explicitly
imported.
However, this also requires duplicating any macros from other sources
(such as std and `tokio-trace-core`) with wrappers due to the behaviour
of `local_inner_macros`. I've added these wrapper macros as well.
Since the macros got even longer as a result of this, I've moved them
to a separate file to make `lib.rs` easier to read. I've also wrapped
some very long lines in the macros, and removed the explicit drop of
the result of evaluating some event macros (it's no longer necessary
as all event macros now evaluate to `()`).
[`local_inner_macros`]: https://doc.rust-lang.org/nightly/edition-guide/rust-2018/macros/macro-changes.html#local-helper-macrosFixes#968
Signed-off-by: Eliza Weisman <[email protected]>
This branch makes a handful of `tokio-trace-core` API improvements, mostly
around naming. In particular:
* Rename `dispatcher::with` to `dispatcher::get_default`
* Rename `Event::observe` to `Event::dispatch`
* Make `field::ValidLen` trait private
Closes#948Closes#960
Signed-off-by: Eliza Weisman <[email protected]>
This branch changes the `Subscriber::record` method to take a new
arguments struct, `span::Record`. The `field::Record` trait was renamed
to `field::Visit` to prevent name conflicts.
In addition, the `ValueSet::is_empty`, `ValueSet::contains`, and
`ValueSet::record` methods were made crate-private, as they are exposed
on the `Attributes` and `Record` types.
Signed-off-by: Eliza Weisman <[email protected]>
This branch allows users of `tokio-trace` to explicitly set a span's
parent, or indicate that a span should be a new root of its own trace
tree. A `parent: ` key has been added to the `span!` macros. When a span
is provided, that span will be set as the parent, while `parent: None`
will result in a new root span. No `parent:` key results in the current
behaviour.
A new type, `span::Attributes`, was added to `tokio-trace-core` to act
as an arguments struct for the `Subscriber::new_span` method. This will
allow future fields to be added without causing breaking API changes.
The `Attributes` struct currently contains the new span's metadata,
`ValueSet`, and parent.
Finally, the `span::Span` type in `-core` was renamed to `span::Id`, for
consistency with `tokio-trace` and to differentiate it from
`span::Attributes`. This name was chosen primarily due to precedent in
other tracing systems.
Closes#920
Signed-off-by: Eliza Weisman <[email protected]>
This branch adds links to the master RustDoc published by CI to the
`tokio-trace` and `tokio-trace-core` README. In addition, it fixes a
broken links in the RustDoc for `tokio-trace` and updates the
`tokio-trace-core` RustDoc to match the README.
Signed-off-by: Eliza Weisman <[email protected]>
## Motivation
As described in #905, subscribers have no way to get the numeric value of a span ID back _out_ of a `Span`.
## Solution
Add a `Span::into_u64` method that returns the inner `u64` span ID
Closes#905
Signed-off-by: kleimkuhler <[email protected]>
<!-- Thank you for your Pull Request. Please provide a description above
and review the requirements below.
Bug fixes and new features should include tests.
Contributors guide:
https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md -->
## Motivation
In asynchronous systems like Tokio, interpreting traditional log
messages can often be quite challenging. Since individual tasks are
multiplexed on the same thread, associated events and log lines are
intermixed making it difficult to trace the logic flow. Currently, none
of the available logging frameworks or libraries in Rust offer the
ability to trace logical paths through a futures-based program.
There also are complementary goals that can be accomplished with such a
system. For example, metrics / instrumentation can be tracked by
observing emitted events, or trace data can be exported to a distributed
tracing or event processing system.
In addition, it can often be useful to generate this diagnostic data in
a structured manner that can be consumed programmatically. While prior
art for structured logging in Rust exists, it is not currently
standardized, and is not "Tokio-friendly".
## Solution
This branch adds a new library to the tokio project, `tokio-trace`.
`tokio-trace` expands upon logging-style diagnostics by allowing
libraries and applications to record structured events with additional
information about *temporality* and *causality* --- unlike a log
message, a span in `tokio-trace` has a beginning and end time, may be
entered and exited by the flow of execution, and may exist within a
nested tree of similar spans. In addition, `tokio-trace` spans are
*structured*, with the ability to record typed data as well as textual
messages.
The `tokio-trace-core` crate contains the core primitives for this
system, which are expected to remain stable, while `tokio-trace` crate
provides a more "batteries-included" API. In particular, it provides
macros which are a superset of the `log` crate's `error!`, `warn!`,
`info!`, `debug!`, and `trace!` macros, allowing users to begin the
process of adopting `tokio-trace` by performing a drop-in replacement.
## Notes
Work on this project had previously been carried out in the
[tokio-trace-prototype] repository. In addition to the `tokio-trace` and
`tokio-trace-core` crates, the `tokio-trace-prototype` repo also
contains prototypes or sketches of adapter, compatibility, and utility
crates which provide useful functionality for `tokio-trace`, but these
crates are not yet ready for a release. When this branch is merged, that
repository will be archived, and the remaining unstable crates will be
moved to a new `tokio-trace-nursery` repository. Remaining issues on the
`tokio-trace-prototype` repo will be moved to the appropriate new repo.
The crates added in this branch are not _identical_ to the current head
of the `tokio-trace-prototype` repo, as I did some final clean-up and docs
polish in this branch prior to merging this PR.
[tokio-trace-prototype]: https://github.com/hawkw/tokio-trace-prototypeCloses: #561
Signed-off-by: Eliza Weisman <[email protected]>
Eliza Weisman