Introduce tokio-trace (#827)

<!-- Thank you for your Pull Request. Please provide a description above
and review the requirements below.

Bug fixes and new features should include tests.

Contributors guide:
https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md -->

## Motivation

In asynchronous systems like Tokio, interpreting traditional log
messages can often be quite challenging. Since individual tasks are
multiplexed on the same thread, associated events and log lines are
intermixed making it difficult to trace the logic flow. Currently, none
of the available logging frameworks or libraries in Rust offer the
ability to trace logical paths through a futures-based program.

There also are complementary goals that can be accomplished with such a
system. For example, metrics / instrumentation can be tracked by
observing emitted events, or trace data can be exported to a distributed
tracing or event processing system.

In addition, it can often be useful to generate this diagnostic data in
a structured manner that can be consumed programmatically. While prior
art for structured logging in Rust exists, it is not currently
standardized, and is not "Tokio-friendly".

## Solution

This branch adds a new library to the tokio project, `tokio-trace`.
`tokio-trace` expands upon logging-style diagnostics by allowing
libraries and applications to record structured events with additional
information about *temporality* and *causality* --- unlike a log
message, a span in `tokio-trace` has a beginning and end time, may be
entered and exited by the flow of execution, and may exist within a
nested tree of similar spans. In addition, `tokio-trace` spans are
*structured*, with the ability to record typed data as well as textual
messages.

The `tokio-trace-core` crate contains the core primitives for this
system, which are expected to remain stable, while `tokio-trace` crate
provides a more "batteries-included" API. In particular, it provides
macros which are a superset of the `log` crate's `error!`, `warn!`,
`info!`, `debug!`, and `trace!` macros, allowing users to begin the
process of adopting `tokio-trace` by performing a drop-in replacement.

## Notes

Work on this project had previously been carried out in the
[tokio-trace-prototype] repository. In addition to the `tokio-trace` and
`tokio-trace-core` crates, the `tokio-trace-prototype` repo also
contains prototypes or sketches of adapter, compatibility, and utility
crates which provide useful functionality for `tokio-trace`, but these
crates are not yet ready for a release. When this branch is merged, that
repository will be archived, and the remaining unstable crates will be
moved to a new `tokio-trace-nursery` repository. Remaining issues on the
`tokio-trace-prototype` repo will be moved to the appropriate new repo.

The crates added in this branch are not _identical_ to the current head
of the `tokio-trace-prototype` repo, as I did some final clean-up and docs
polish in this branch prior to merging this PR.

[tokio-trace-prototype]: https://github.com/hawkw/tokio-trace-prototype

Closes: #561

Signed-off-by: Eliza Weisman <[email protected]>

tokio-trace/tokio-trace-core/src/metadata.rs

@@ -0,0 +1,272 @@
+//! Metadata describing trace data.
+use super::{
+    callsite::{self, Callsite},
+    field,
+};
+use std::fmt;
+
+/// Metadata describing a [`Span`].
+///
+/// This includes the source code location where the span occurred, the names of
+/// its fields, et cetera.
+///
+/// Metadata is used by [`Subscriber`]s when filtering spans and events, and it
+/// may also be used as part of their data payload.
+///
+/// When created by the `event!` or `span!` macro, the metadata describing a
+/// particular event or span is constructed statically and exists as a single
+/// static instance. Thus, the overhead of creating the metadata is
+/// _significantly_ lower than that of creating the actual span. Therefore,
+/// filtering is based on metadata, rather than  on the constructed span.
+///
+/// **Note**: Although instances of `Metadata` cannot be compared directly, they
+/// provide a method [`Metadata::id()`] which returns an an opaque [callsite
+/// identifier] which uniquely identifies the callsite where the metadata
+/// originated. This can be used for determining if two Metadata correspond to
+/// the same callsite.
+///
+/// [`Span`]: ::span::Span
+/// [`Subscriber`]: ::Subscriber
+/// [`Metadata::id()`]: ::metadata::Metadata::id
+/// [callsite identifier]: ::callsite::Identifier
+// TODO: When `const fn` is stable, make this type's fields private.
+pub struct Metadata<'a> {
+    /// The name of the span described by this metadata.
+    ///
+    /// **Warning**: The fields on this type are currently `pub` because it must
+    /// be able to be constructed statically by macros. However, when `const
+    /// fn`s are available on stable Rust, this will no longer be necessary.
+    /// Thus, these fields are *not* considered stable public API, and they may
+    /// change warning. Do not rely on any fields on `Metadata`. When
+    /// constructing new `Metadata`, use the `metadata!` macro or the
+    /// `Metadata::new` constructor instead!
+    #[doc(hidden)]
+    pub name: &'static str,
+
+    /// The part of the system that the span that this metadata describes
+    /// occurred in.
+    ///
+    /// Typically, this is the module path, but alternate targets may be set
+    /// when spans or events are constructed.
+    ///
+    /// **Warning**: The fields on this type are currently `pub` because it must
+    /// be able to be constructed statically by macros. However, when `const
+    /// fn`s are available on stable Rust, this will no longer be necessary.
+    /// Thus, these fields are *not* considered stable public API, and they may
+    /// change warning. Do not rely on any fields on `Metadata`. When
+    /// constructing new `Metadata`, use the `metadata!` macro or the
+    /// `Metadata::new` constructor instead!
+    #[doc(hidden)]
+    pub target: &'a str,
+
+    /// The level of verbosity of the described span.
+    ///
+    /// **Warning**: The fields on this type are currently `pub` because it must
+    /// be able to be constructed statically by macros. However, when `const
+    /// fn`s are available on stable Rust, this will no longer be necessary.
+    /// Thus, these fields are *not* considered stable public API, and they may
+    /// change warning. Do not rely on any fields on `Metadata`. When
+    /// constructing new `Metadata`, use the `metadata!` macro or the
+    /// `Metadata::new` constructor instead!
+    #[doc(hidden)]
+    pub level: Level,
+
+    /// The name of the Rust module where the span occurred, or `None` if this
+    /// could not be determined.
+    ///
+    /// **Warning**: The fields on this type are currently `pub` because it must
+    /// be able to be constructed statically by macros. However, when `const
+    /// fn`s are available on stable Rust, this will no longer be necessary.
+    /// Thus, these fields are *not* considered stable public API, and they may
+    /// change warning. Do not rely on any fields on `Metadata`. When
+    /// constructing new `Metadata`, use the `metadata!` macro or the
+    /// `Metadata::new` constructor instead!
+    #[doc(hidden)]
+    pub module_path: Option<&'a str>,
+
+    /// The name of the source code file where the span occurred, or `None` if
+    /// this could not be determined.
+    ///
+    /// **Warning**: The fields on this type are currently `pub` because it must
+    /// be able to be constructed statically by macros. However, when `const
+    /// fn`s are available on stable Rust, this will no longer be necessary.
+    /// Thus, these fields are *not* considered stable public API, and they may
+    /// change warning. Do not rely on any fields on `Metadata`. When
+    /// constructing new `Metadata`, use the `metadata!` macro or the
+    /// `Metadata::new` constructor instead!
+    #[doc(hidden)]
+    pub file: Option<&'a str>,
+
+    /// The line number in the source code file where the span occurred, or
+    /// `None` if this could not be determined.
+    ///
+    /// **Warning**: The fields on this type are currently `pub` because it must
+    /// be able to be constructed statically by macros. However, when `const
+    /// fn`s are available on stable Rust, this will no longer be necessary.
+    /// Thus, these fields are *not* considered stable public API, and they may
+    /// change warning. Do not rely on any fields on `Metadata`. When
+    /// constructing new `Metadata`, use the `metadata!` macro or the
+    /// `Metadata::new` constructor instead!
+    #[doc(hidden)]
+    pub line: Option<u32>,
+
+    /// The names of the key-value fields attached to the described span or
+    /// event.
+    ///
+    /// **Warning**: The fields on this type are currently `pub` because it must
+    /// be able to be constructed statically by macros. However, when `const
+    /// fn`s are available on stable Rust, this will no longer be necessary.
+    /// Thus, these fields are *not* considered stable public API, and they may
+    /// change warning. Do not rely on any fields on `Metadata`. When
+    /// constructing new `Metadata`, use the `metadata!` macro or the
+    /// `Metadata::new` constructor instead!
+    #[doc(hidden)]
+    pub fields: field::FieldSet,
+}
+
+/// Describes the level of verbosity of a `Span`.
+#[derive(Clone, Debug, Eq, PartialEq, Ord, PartialOrd)]
+pub struct Level(LevelInner);
+
+// ===== impl Metadata =====
+
+impl<'a> Metadata<'a> {
+    /// Construct new metadata for a span, with a name, target, level, field
+    /// names, and optional source code location.
+    pub fn new(
+        name: &'static str,
+        target: &'a str,
+        level: Level,
+        module_path: Option<&'a str>,
+        file: Option<&'a str>,
+        line: Option<u32>,
+        field_names: &'static [&'static str],
+        callsite: &'static Callsite,
+    ) -> Self {
+        Metadata {
+            name,
+            target,
+            level,
+            module_path,
+            file,
+            line,
+            fields: field::FieldSet {
+                names: field_names,
+                callsite: callsite::Identifier(callsite),
+            },
+        }
+    }
+
+    /// Returns the set of fields on the described span.
+    pub fn fields(&self) -> &field::FieldSet {
+        &self.fields
+    }
+
+    /// Returns the level of verbosity of the described span.
+    pub fn level(&self) -> &Level {
+        &self.level
+    }
+
+    /// Returns the name of the span.
+    pub fn name(&self) -> &'static str {
+        self.name
+    }
+
+    /// Returns a string describing the part of the system where the span or
+    /// event that this metadata describes occurred.
+    ///
+    /// Typically, this is the module path, but alternate targets may be set
+    /// when spans or events are constructed.
+    pub fn target(&self) -> &'a str {
+        self.target
+    }
+
+    /// Returns the path to the Rust module where the span occurred, or
+    /// `None` if the module path is unknown.
+    pub fn module_path(&self) -> Option<&'a str> {
+        self.module_path
+    }
+
+    /// Returns the name of the source code file where the span
+    /// occurred, or `None` if the file is unknown
+    pub fn file(&self) -> Option<&'a str> {
+        self.file
+    }
+
+    /// Returns the line number in the source code file where the span
+    /// occurred, or `None` if the line number is unknown.
+    pub fn line(&self) -> Option<u32> {
+        self.line
+    }
+
+    /// Returns an opaque `Identifier` that uniquely identifies the callsite
+    /// this `Metadata` originated from.
+    #[inline]
+    pub fn callsite(&self) -> callsite::Identifier {
+        self.fields.callsite()
+    }
+}
+
+impl<'a> fmt::Debug for Metadata<'a> {
+    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
+        f.debug_struct("Metadata")
+            .field("name", &self.name)
+            .field("target", &self.target)
+            .field("level", &self.level)
+            .field("module_path", &self.module_path)
+            .field("file", &self.file)
+            .field("line", &self.line)
+            .field("field_names", &self.fields)
+            .finish()
+    }
+}
+
+// ===== impl Level =====
+
+impl Level {
+    /// The "error" level.
+    ///
+    /// Designates very serious errors.
+    pub const ERROR: Level = Level(LevelInner::Error);
+    /// The "warn" level.
+    ///
+    /// Designates hazardous situations.
+    pub const WARN: Level = Level(LevelInner::Warn);
+    /// The "info" level.
+    ///
+    /// Designates useful information.
+    pub const INFO: Level = Level(LevelInner::Info);
+    /// The "debug" level.
+    ///
+    /// Designates lower priority information.
+    pub const DEBUG: Level = Level(LevelInner::Debug);
+    /// The "trace" level.
+    ///
+    /// Designates very low priority, often extremely verbose, information.
+    pub const TRACE: Level = Level(LevelInner::Trace);
+}
+
+#[repr(usize)]
+#[derive(Copy, Clone, Debug, Eq, PartialEq, Hash, Ord, PartialOrd)]
+enum LevelInner {
+    /// The "error" level.
+    ///
+    /// Designates very serious errors.
+    Error = 1,
+    /// The "warn" level.
+    ///
+    /// Designates hazardous situations.
+    Warn,
+    /// The "info" level.
+    ///
+    /// Designates useful information.
+    Info,
+    /// The "debug" level.
+    ///
+    /// Designates lower priority information.
+    Debug,
+    /// The "trace" level.
+    ///
+    /// Designates very low priority, often extremely verbose, information.
+    Trace,
+}