1. Basic Information

2. Key Takeaways

Researchers re-registered an expired domain previously used for contact by Kokoro developers and used password reset functionality to take over the Hugging Face account. They then fine-tuned the model weights locally in 87 seconds, causing the model to generate audio that differed from the input text.

3. Research Objectives

  • To examine the extent to which open model registries can guarantee the continuity of publisher identity and the semantic integrity of model weights.

4. Methodology and Workflow

  1. Verify that the contact domain listed in the public repository has expired.
  2. Re-register the domain and receive the password reset email sent to the Hugging Face publisher account.
  3. Take control of the account.
  4. Fine-tune the Kokoro model weights locally to alter the output audio without modifying the application code.
  5. Refrain from uploading the modified model weights to Hugging Face due to safety risks.

5. Inputs and Outputs

  • Inputs: Expired domain, publisher account, Kokoro model weights, and input text.
  • Outputs: Account control and a modified model that speaks content different from the input text using the same inference pipeline.

6. Evaluation Design

  • The researchers used controlled testing to assess whether they could take over the account and alter model behavior without modifying the consuming application.

7. Results

  • Acquisition of the publisher account via password reset was successful.
  • Local testing showed altered audio output after 87 seconds of fine-tuning using a gaming GPU.
  • The researchers did not distribute the modified weights through the registry or test automatic updates in user environments; no resulting harm in production was confirmed.

8. Practical Implications

  • The researchers explain that tampering with model weights that contain no executable code cannot be detected by malware signatures alone. Pinning to known trusted revisions or hash values helps detect unauthorized replacements, but does not guarantee safety if a compromised publisher signs a new release or if new hash values are trusted blindly. Furthermore, validation via behavioral regression tests remains limited to the range of tested inputs.

9. Conditions, Limitations, and Scope

  • The preconditions are that the expired domain is used for password reset emails and that the registry does not require additional ownership verification.
  • This work is controlled research and does not demonstrate the distribution of malicious models or actual harm to Kokoro users.

10. Facts, Inferences, and Hypotheses

Facts

  • Researchers registered hexgrad.com and used it for publisher account password recovery.
  • Kokoro is reported to have approximately 13 million monthly downloads and is used in guides and tools by multiple vendors.
  • Modified weights were not uploaded to Hugging Face.

Inferences

  • Consumers tracking the main branch or performing automatic downloads could receive modified weights from legitimate channels if the publisher account is compromised.

Hypotheses

No additional hypotheses. Unverified items are listed under 'Open Questions and Further Investigation'.

11. MITRE ATT&CK Mapping

ID Technique Confidence Basis
T1583.001 Acquire Infrastructure: Domains medium Researchers acquired a domain to use in account recovery. No actual criminal activity was observed. Source

12. Open Questions and Further Investigation

  • Researchers stated that they secured the account and are coordinating with Hugging Face. Return to the original owner and permanent registry-side mitigations have not been detailed.
  • The monthly download figure of approximately 13 million reported by BleepingComputer differs from the roughly 19 million pulls reported by the researcher for the previous month. It remains unverified whether the aggregation periods and criteria match.
  • The impact on mirrors and converted models is unverified.

13. Implications for Defenders

Organizations deploying AI models in production should record model names, commit hashes, file hashes, signatures, sources, and publisher contact domains in an SBOM-equivalent ledger, and compare outputs before and after model updates.

14. Summary by Target Audience

  • SOC Teams: Monitor for model acquisition and updates, changes in hash values, registry account modifications, and anomalies in inference outputs.
  • Administrators: Implement revision pinning and artifact verification, and ensure organization-owned domains are used for account recovery.
  • Users: Researchers have not distributed modified model weights, and this report does not indicate successful compromise of Kokoro users.