Skip to content
This repository was archived by the owner on May 1, 2025. It is now read-only.
This repository was archived by the owner on May 1, 2025. It is now read-only.

MPS support #40

@tmm1

Description

@tmm1

It would be nice to support Metal Performance Shaders backend for pytorch on macOS

the main changes for inference seem to be:

  • load_in_8bit must be False
  • device_map set to {"":"mps"}

see:

https://github.com/Birch-san/falcon-play/blob/cbf9b2aebe7eef3eea305a511d6cdda17282ca8a/scripts/chat_play.py#L154-L157

https://github.com/Birch-san/falcon-play/blob/cbf9b2aebe7eef3eea305a511d6cdda17282ca8a/scripts/chat_play.py#L175

cc @Birch-san are there any other good docs or resources for adding mps support?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions