mirror of
https://github.com/tokio-rs/tokio.git
synced 2026-08-29 00:00:11 +02:00
b2c53987d9af8cdaa6ba49a084bb4363f86bd295
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Eliza Weisman
|
b2c53987d9 |
trace: Add shorthand for field::display and field::debug (#1088)
## Motivation In `tokio-trace`, field values may be recorded as either a subset of Rust primitive types or as `fmt::Display` and `fmt::Debug` implementations. Currently, `tokio-trace` provides the `field::display` and `field::debug` functions which wrap a type with a type that implements `Value` using the wrapped type's `fmt::Display` or `fmt::Debug` implementation. However, importing and using these functions adds unnecessary boilerplate. In #1081, @jonhoo suggested adding shorthand syntax to the macros, similar to that used by the `slog` crate, as a solution for the wordiness of the current API. ## Solution This branch adds `?` and `%` sigils to field values in the span and event macros, which expand to the `field::debug` and `field::display` wrappers, respectively. The shorthand sigils may be used in any position where the macros take a field value. For example: ```rust trace_span!("foo", my_field = ?something, ...); // shorthand for `debug` info!(foo = %value, bar = false, ...) // shorthand for `display` ``` Adding this shorthand required a fairly large change to how field key-value pairs are handled by the macros --- since `%foo` and `%foo` are not valid Rust expressions, we can no longer match repeated `$ident = $expr` patterns, and must now match field lists as repeated token trees. The inner helper macros for constructing `FieldSet`s and `ValueSet`s have to parse the token trees recursively. This added a decent chunk of complexity, but fortunately we have a large number of compile tests for the macros and I'm quite confident that all existing invocations will still work. Closes #1081 Signed-off-by: Eliza Weisman <[email protected]> |
||
|
Eliza Weisman
|
197f88f3bc |
trace: Change Span::enter and record to take &self (#1029)
## Motivation
The `Span::enter` function previously required an `&mut` reference to
enter a span. This is a relic of an earlier design where span closure
logic was determined by dropping an inner span component, and is no
longer strictly necessary.
Requiring `&mut self` to enter a span leads to awkward patterns in cases
when a user wishes to enter a span and then call methods on the span
(such as recording field values). For example, we cannot say
```rust
let mut span = span!("foo", bar);
span.enter(|| {
span.record("bar" &false);
});
```
since the span is mutably borrowed by `enter`. Instead, we must clone
the span, like so:
```rust
let mut span = span!("foo", bar);
span.clone().enter(|| {
span.record("bar" &false);
});
```
Having to clone the span is somewhat less ergonomic, and it has
performance disadvantages as well: cloning a `Span` will clone the
span's `Dispatch` handle, requiring an `Arc` bump, as well as calling
the `Subscriber`'s `clone_span` and `drop_span` functions. If we can
enter spans without a mutable borrow, we don't have to update any of
these ref counts.
The other reason we may wish to require mutable borrows to enter a span
is if we want to disallow entering a span multiple times before exiting
it. However, it is trivially possible to re-enter a span on the same
thread regardless, by cloning the span and entering it twice. Besides,
there may be a valuable semantic meaning in entering a span from inside
itself, such as when a function is called recursively, so disallowing
this is not a goal.
## Solution
This branch rewrites the `Span::enter`, `Span::record`, and
`Span::record_all` functions to no longer require mutable borrows.
In the case of `record` and `record_all`, this was trivial, as borrowing
mutably was not actually *necessary* for those functions. For `enter`,
the `Entered` guard type was reworked to consist of an `&'a Inner`
rather than an `Inner`, so it is no longer necessary to `take` the
span's `Inner`.
## Notes
In addition to allowing spans to be entered without mutable borrows,
`Entered` was changed to exit the span automatically when the guard is
dropped, so we may now observe correct span exits even when unwinding.
Furthermore, this allows us to simplify the `enter` function a bit,
leading to a minor performance improvement when entering spans.
Before:
```
test enter_span ... bench: 13 ns/iter (+/- 1)
```
...and after:
```
test enter_span ... bench: 3 ns/iter (+/- 1)
```
Note that this branch also contains a change to make the
`subscriber::enter_span` benchmark more accurate. Previously, this
benchmark constructed a new span inside of `b.iter(|| {...})`. This
means that the benchmark was measuring not only the time taken to enter
a span, but the time taken to construct a `Span` handle as well.
However, we already have benchmarks for span construction, and the
intention of this particular benchmark was to measure the overhead of
constructing a span.
I've updated the benchmark by moving the span construction out of the
`iter` closure. Now, the span is constructed a single time and entered
on every iteration. This allows us to measure only the overhead of
actually entering a span. The "before" benchmark numbers above were
recorded after backporting this change to master, so they are "fair" to
the previous implementation. Prior to this change the benchmark took
approximately 53 ns.
Signed-off-by: Eliza Weisman <[email protected]>
|
||
|
Eliza Weisman
|
44f65afcc6 |
trace: Allow field names to be separated by .s (#1027)
## Motivation In order to support conventions that add namespacing to `tokio-trace` field names, it's necessary to accept at least one type of separator character. Currently, the `tokio-trace` macros only accept valid Rust identifiers, so there is no clear separator character for namespaced conventions. See also #1018. ## Solution This branch changes the single `ident` fragment matcher for field names to match *one or more* `ident` fragments separated by `.` characters. ## Notes The resulting key is still exposed to `tokio-trace-core` as a string constant created by stringifying the dotted expression. However, if `tokio-trace-core` were later to adopt a first class notion of hierarchical field keys, we would be able to track that change in `tokio-trace` as an implementation detail. Closes #1018. Closes #1022. Signed-off-by: Eliza Weisman <[email protected]> |
||
|
João Oliveira
|
597f271c08 |
trace: Remove default trace level and make levels mandatory on span! macro (#1025)
## Motivation Was determined that having the span! macro default to the TRACE level is probably not ideal (see discussion on #952). Closes #1013 ## Solution Remove default trace level and make log lvl mandatory on span! macro, and add the respective `trace_span!`, `debug_span!`, `info_span!`, `warn_span!` and `error_span!` macros that behave as span! macro, but with defined log levels ## Notes I think this is it, also removed some captures that were repeated, and some testcases that also seemed repeated after adding the mandatory log level, but please review it, if more tests or examples are needed happy to provide (tried to find a way to get the generated macros log level, but didn't find one, if there is a way i can add tests to assert that the generated macro has the matching log level ). thanks |
||
|
Eliza Weisman
|
d8177f81ac |
trace: Allow trace instrumentation to emit log records (#992)
## Motivation `tokio-trace` currently offers a strategy for compatibility with the `log` crate: its macros can be dropped in as a replacement for `log`'s macros, and a subscriber can be used that translates trace events to log records. However, this requires the application to be aware of `tokio-trace` and manually set up this subscriber. Many libraries currently emit `log` records, and would like to be able to emit `tokio-trace` instrumentation instead. The `tokio` runtimes are one such example. However, with the current log compatibility strategy, replacing existing logging with trace instrumentation would break `tokio`'s logs for any downstream user which is using only `log` and not `tokio-trace`. It is desirable for libraries to have the option to emit both `log` _and_ `tokio-trace` diagnostics from the same instrumentation points. ## Solution This branch adds a `log` feature flag to the `tokio-trace` crate, which when set, causes `tokio-trace` instrumentation to emit log records as well as `tokio-trace` instrumentation. ## Notes In order to allow spans to log their names when they are entered and exited even when the span is disabled, this branch adds an `&'static Metadata` to the `Span` type. This was previously stored in the `Inner` type and was thus only present when the span was enabled. This makes disabled spans one word longer, but enabled spans remain the same size. Fixes: #949 Signed-off-by: Eliza Weisman <[email protected]> |
||
|
Eliza Weisman
|
85487727d4 |
trace: Span API polish (#988)
This branch makes the following changes to `tokio-trace`'s `Span` type: * **Remove manual close API from spans** In practice, there wasn't really a use-case for this, and it complicates the implementation a bit. We can always add it back later. * **Remove generic lifetime from `Span`** Again, there wasn't actually a use-case for spans with metadata that doesn't live for the static lifetime, and it made using `Span`s in other types somewhat inconvenient. It's also possible to implement an alternative API for non-static spans on top of the `tokio-trace-core` primitives. Signed-off-by: Eliza Weisman <[email protected]> |
||
|
Eliza Weisman
|
b8f63308d7 |
trace-core: Pass dispatcher by ref to dispatcher::with_default (#971)
* trace-core: Pass dispatcher by ref to `dispatcher::with_default` As requested by @carllerche in https://github.com/tokio-rs/tokio/pull/966#discussion_r264380005, this branch changes the `dispatcher::with_default` function in `tokio-trace-core` to take the dispatcher by ref and perform the clone internally. This makes this function more consistant with other `with_default` functions in other crates. Signed-off-by: Eliza Weisman <[email protected]> * trace: Don't set the default dispatcher on entering a span Setting the default dispatcher on span entry is a relic of when spans tracked their parent's ID. At that time, it was necessary to ensure that any spans created inside a span were observed by the same subscriber that originally provided the entered span with an ID, as otherwise, new spans would be created with parent IDs that did not originate from that subscriber. Now that spans don't track their parent ID, this is no longer necessary. However, removing this behavior does mean that if a span is entered outside of the subscriber context it was created in, any subsequent spans will be observed by the current default subscriber and thus will not be part of the original span's trace tree. Since subscribers are not expected to change frequently, and spans are not expected to move between them, this is likely acceptable. I've removed the tests for the old behavior. Note that this change improves the performance of span entry/exit fairly significantly. Here are the results of running a benchmark that enters a span, does nothing, and immediately exits it, before this change: ``` test enter_span ... bench: 93 ns/iter (+/- 14) ``` ...and after: ``` test enter_span ... bench: 51 ns/iter (+/- 9) ``` Signed-off-by: Eliza Weisman <[email protected]> |
||
|
Eliza Weisman
|
5ff6e37c59 |
trace: Allow specifying a new span's parent (#923)
This branch allows users of `tokio-trace` to explicitly set a span's parent, or indicate that a span should be a new root of its own trace tree. A `parent: ` key has been added to the `span!` macros. When a span is provided, that span will be set as the parent, while `parent: None` will result in a new root span. No `parent:` key results in the current behaviour. A new type, `span::Attributes`, was added to `tokio-trace-core` to act as an arguments struct for the `Subscriber::new_span` method. This will allow future fields to be added without causing breaking API changes. The `Attributes` struct currently contains the new span's metadata, `ValueSet`, and parent. Finally, the `span::Span` type in `-core` was renamed to `span::Id`, for consistency with `tokio-trace` and to differentiate it from `span::Attributes`. This name was chosen primarily due to precedent in other tracing systems. Closes #920 Signed-off-by: Eliza Weisman <[email protected]> |
||
|
Eliza Weisman
|
c08e73c8d4 |
Introduce tokio-trace (#827)
<!-- Thank you for your Pull Request. Please provide a description above and review the requirements below. Bug fixes and new features should include tests. Contributors guide: https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md --> ## Motivation In asynchronous systems like Tokio, interpreting traditional log messages can often be quite challenging. Since individual tasks are multiplexed on the same thread, associated events and log lines are intermixed making it difficult to trace the logic flow. Currently, none of the available logging frameworks or libraries in Rust offer the ability to trace logical paths through a futures-based program. There also are complementary goals that can be accomplished with such a system. For example, metrics / instrumentation can be tracked by observing emitted events, or trace data can be exported to a distributed tracing or event processing system. In addition, it can often be useful to generate this diagnostic data in a structured manner that can be consumed programmatically. While prior art for structured logging in Rust exists, it is not currently standardized, and is not "Tokio-friendly". ## Solution This branch adds a new library to the tokio project, `tokio-trace`. `tokio-trace` expands upon logging-style diagnostics by allowing libraries and applications to record structured events with additional information about *temporality* and *causality* --- unlike a log message, a span in `tokio-trace` has a beginning and end time, may be entered and exited by the flow of execution, and may exist within a nested tree of similar spans. In addition, `tokio-trace` spans are *structured*, with the ability to record typed data as well as textual messages. The `tokio-trace-core` crate contains the core primitives for this system, which are expected to remain stable, while `tokio-trace` crate provides a more "batteries-included" API. In particular, it provides macros which are a superset of the `log` crate's `error!`, `warn!`, `info!`, `debug!`, and `trace!` macros, allowing users to begin the process of adopting `tokio-trace` by performing a drop-in replacement. ## Notes Work on this project had previously been carried out in the [tokio-trace-prototype] repository. In addition to the `tokio-trace` and `tokio-trace-core` crates, the `tokio-trace-prototype` repo also contains prototypes or sketches of adapter, compatibility, and utility crates which provide useful functionality for `tokio-trace`, but these crates are not yet ready for a release. When this branch is merged, that repository will be archived, and the remaining unstable crates will be moved to a new `tokio-trace-nursery` repository. Remaining issues on the `tokio-trace-prototype` repo will be moved to the appropriate new repo. The crates added in this branch are not _identical_ to the current head of the `tokio-trace-prototype` repo, as I did some final clean-up and docs polish in this branch prior to merging this PR. [tokio-trace-prototype]: https://github.com/hawkw/tokio-trace-prototype Closes: #561 Signed-off-by: Eliza Weisman <[email protected]> |