Abstract
This disclosure describes techniques and metrics that can be utilized for evaluation of the quality of streaming translations. Per techniques of this disclosure, a time lag family of metrics and an erasure time lag family of metrics are utilized to characterize and/or compare the performance of machine translation techniques utilized in generating streaming translation. The latency of a translation session is measured by a time lag family of metrics. A time lag is the period of time by which each token in a response log lags behind a corresponding token in a query log. Erasure time lag is similar to the time lag family of metrics but changes the notion of token timing to account for output stability. Whereas time lag metrics track the timestamp of a token’s first appearance, erasure time metrics track the timestamp where a token and all tokens before it became stable in the output.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Cherry, Colin; Ye, Tianjun; Arivazhagan, Naveen; I, Te; He, Yingjie; Shi, Yue; Li, Yuezhang; and Tian, Yuan, "Metrics for Streaming Translation", Technical Disclosure Commons, (May 31, 2022)
https://www.tdcommons.org/dpubs_series/5177