Skip to content

Commit

Permalink
Merge pull request #2 from jinze1994/main
Browse files Browse the repository at this point in the history
Update README about News
  • Loading branch information
jinze1994 committed Dec 27, 2022
2 parents 03fb2c0 + dc93ecd commit f09a3f5
Showing 1 changed file with 24 additions and 1 deletion.
25 changes: 24 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,11 @@ For now, OFASys supports 7 modalities and more than 20 classes of multi-modal ta
* Structural Language: for tasks like Text-to-SQL, Table-to-Text, Table question answering, and Sudoku.
* Motion: for tasks like Text-to-Motion.

# News
* 2022.12.23 v0.1.0-patch1:
- Refactored and released diffusion-based `Text-to-Motion` task (v0.1), see [doc](https://ofasys-doc.readthedocs.io/en/latest/task/motion.html) for usage.
- Refactored TextPreprocess: BOS and EOS no longer required when writing an instruction.
- Added DatabasePreprocess for the `Text-to-SQL` task.

# Requirements

Expand Down Expand Up @@ -248,7 +253,25 @@ output[0].save_image('0.png')

<img src="https://ofasys.oss-cn-zhangjiakou.aliyuncs.com/examples/image-gen_example.png" width="400">


### Text-to-Motion Generation

```
model = OFASys.from_pretrained('single_task_motion.pt')
instruction = 'motion capture: [TEXT:text] -> [MOTION:bvh_frames,preprocess=motion_6d,adaptor=motion_6d]'
guided_prompts = [
{'text': 'run then jump'}, # # The positive prompt.
{'text': ''}, # The negative prompt, or an empty string for classifier-free guidance.
]
# This API requires the positive and negative prompts be in the same batch, so please ensure batch_size % 2 == 0.
output = model.inference(instruction, data=guided_prompts, guidance_weight=3.0, batch_size=2)
output[0].save_as_gif('run_then_jump__guided.gif')
```

<img src="https://ofasys.oss-cn-zhangjiakou.aliyuncs.com/examples/run_then_jump_guided.gif" width="400">

The checkpoint of the single motion task and more motion cases can be found at [here](https://ofasys-doc.readthedocs.io/en/latest/task/motion.html).


# Learn More

| Section | Description |
Expand Down

0 comments on commit f09a3f5

Please sign in to comment.