Amazon has issued a memo to its software engineers, addressing the use of artificial intelligence (AI) tools and particularly discouraging the practice known as "tokenmaxxing." The memo, circulated internally in September and made public on the company's website on Tuesday, outlines new guidelines for the role of software development engineers and managers in the AI era.
Under the leadership of CEO Andy Jassy, Amazon has swiftly integrated AI across its vast business operations. Jassy has previously stated that AI will significantly alter the scope and speed of daily work, potentially reducing the corporate workforce through automation while also creating new job opportunities. However, as employees rush to adopt AI tools, the company faces challenges in managing the technology's escalating costs and assessing its impact on productivity and return on investment.
The memo highlights the pitfalls of tokenmaxxing, a practice where developers maximize AI usage without considering the quality of output. While some companies have imposed limits on token usage to curb rising AI expenses, Amazon has not yet implemented such caps. In May, the company discontinued an internal leaderboard after discovering that employees were engaging in tokenmaxxing to improve their rankings.
Amazon's memo advises that metrics like token usage, AI tool adoption rates, and code changes or reviews are "deeply flawed" indicators of AI usage within teams or organizations. These metrics can lead to increased costs and unnecessary complexity. Instead, the company suggests evaluating these metrics "with care and judgment" alongside other data, emphasizing the importance of focusing on long-term outcomes.
Marc Brooker, a vice president and distinguished engineer at Amazon Web Services, explained in an interview that the company is exploring metrics such as "feature velocity" to assess AI's impact on engineer productivity. This metric considers factors like the number of launches and successful changes in production. Brooker emphasized that the company views this as a return on investment, stating, "If there is an important problem to solve for our customers, important problem to solve for our business, we're going to invest tools, time, tokens, human effort, etc. into those."
Brooker also noted the importance of avoiding wasted effort, saying, "What we don't want to see is wasted effort where we are going and spending those tokens solving problems that don't matter and sometimes it's hard to know, so we do need to have some appetite for experimentation."
The memo further advises employees to refrain from using AI when it does not suit the task at hand. "If your work does not benefit from current AI tooling, that is a finding, not a failing, " the memo states. Amazon encourages engineers to demonstrate judgment by identifying areas where AI is not yet beneficial to their work.
This guidance comes as companies across various industries grapple with the rapid adoption of AI technologies and the associated challenges. While AI holds the promise of transforming business operations, it also presents new complexities in terms of cost management and productivity measurement. Amazon's approach underscores the need for careful consideration and strategic use of AI tools to ensure they deliver meaningful value.
