Python sample client for mimi

Preparation

Use Python 3.10+.

1. Install python library.

   pip install -r requirements.txt

2. Get an access token and write the access token to any file. Before executing access_token.py, do NOT forget to write your ids to the following variables in the file.

   • APPLICATION_ID
   • APPLICATION_SECRET

   python3 access_token.py > token.txt

   • For more information, refer to the Mimi Auth API documentation.

3. Execute speech recognition, machine translation, or speech synthesis.

Execution

Streaming an audio file

An audio file must be raw (headerless linear PCM), 16 bits, and monaural. The sampling rate is your option, but write the rate like rate=16000 in the content-type of headers.

For asr（mimi ASR）:

python3 asr_stream_file.py token.txt audio.raw
#=> { "response" : [ { "pronunciation" : "チョット", "result" : "ちょっと", "time" : [ 580, 1030 ] }, { "pronunciation" : "オソイ", "result" : "遅い", "time" : [ 1030, 1390 ] }, { "pronunciation" : "チューショク", "result" : "昼食", "time" : [ 1390, 1890 ] }, { "pronunciation" : "ヲ", "result" : "を", "time" : [ 1890, 1980 ] }, { "pronunciation" : "トル", "result" : "とる", "time" : [ 1980, 2220 ] }, { "pronunciation" : "タメ", "result" : "ため", "time" : [ 2220, 2610 ] }, { "pronunciation" : "ファミリー", "result" : "ファミリー", "time" : [ 2930, 3480 ] }, { "pronunciation" : "レストラン", "result" : "レストラン", "time" : [ 3480, 4020 ] }, { "pronunciation" : "ニ", "result" : "に", "time" : [ 4020, 4150 ] }, { "pronunciation" : "ハイッ", "result" : "入っ", "time" : [ 4150, 4450 ] }, { "pronunciation" : "タ", "result" : "た", "time" : [ 4450, 4540 ] }, { "pronunciation" : "ノ", "result" : "の", "time" : [ 4540, 4680 ] }, { "pronunciation" : "デス", "result" : "です", "time" : [ 4680, 5100 ] } ], "session_id" : "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX", "status" : "recog-finished", "type" : "asr#mimilvcsr" }

For asr-nict（mimi ASR powered by NICT）:

python3 asr_nict_stream_file.py token.txt audio.raw -f v1
#=> {"type": "asr#nictlvcsr", "session_id": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX", "status": "recog-finished", "response": [{"result": "ちょっと|チョット|ちょっと|副詞-助詞類接続||||"},{"result": "遅い|オソイ|遅い|形容詞-自立|形容詞・アウオ段|基本形||"},{"result": "昼食|チュウショク|昼食|名詞-一般||||"},{"result": "を|ヲ|を|助詞-格助詞-一般||||"},{"result": "とる|トル|とる|動詞-自立|五段・ラ行|基本形||"},{"result": "ため|タメ|ため|名詞-非自立-副詞可能||||"},{"result": "ファミリーレストラン|ファミリーレストラン|ファミリーレストラン|名詞-一般||||"},{"result": "に|ニ|に|助詞-格助詞-一般||||"},{"result": "入っ|ハイッ|入る|動詞-自立|五段・ラ行|連用タ接続||"},{"result": "た|タ|た|助動詞|特殊・タ|基本形||"},{"result": "の|ノ|の|名詞-非自立-一般||||"},{"result": "です|デス|です|助動詞|特殊・デス|基本形||"},{"result": "|||SENT-START-END||||"},{"result": "|||UTT-END||||"}]}

With -f v2 option:

The output can include two kinds of intermediate results, both disabled by default. Use --temporary to enable temporary results ("determined": false) which may still change, or --progressive to enable progressive results ("status": "recog-in-progress" with "determined": true) which are confirmed segments delivered before recognition finishes.

python3 asr_nict_stream_file.py token.txt audio.raw -f v2 --progressive --temporary
#=> {"type":"asr#nictlvcsr2","session_id":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX","status":"recog-in-progress","response":[{"result":"ちょっと遅い。","words":["ちょっと","遅い","。"],"determined":false,"time":0}]}
#=> {"type":"asr#nictlvcsr2","session_id":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX","status":"recog-in-progress","response":[{"result":"ちょっと遅い昼食をとるため。","words":["ちょっと","遅い","昼食","を","とる","ため","。"],"determined":false,"time":0}]}
#=> {"type":"asr#nictlvcsr2","session_id":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX","status":"recog-in-progress","response":[{"result":"ちょっと遅い昼食","words":["ちょっと","遅い","昼食"],"determined":true,"time":0}]}
#=> {"type":"asr#nictlvcsr2","session_id":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX","status":"recog-in-progress","response":[{"result":"をとるためファミリーレストランに","words":["を","とる","ため","ファミリーレストラン","に"],"determined":false,"time":1890}]}
#=> {"type":"asr#nictlvcsr2","session_id":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX","status":"recog-in-progress","response":[{"result":"をとるためファミリーレストランに入ったの","words":["を","とる","ため","ファミリーレストラン","に","入っ","た","の"],"determined":false,"time":1890}]}
#=> {"type":"asr#nictlvcsr2","session_id":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX","status":"recog-in-progress","response":[{"result":"をとるためファミリーレストランに入っ","words":["を","とる","ため","ファミリーレストラン","に","入っ"],"determined":true,"time":1890}]}
#=> {"type":"asr#nictlvcsr2","session_id":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX","status":"recog-finished","response":[{"result":"たのです。","words":["た","の","です"],"determined":true,"time":4440}]}
#=> recog-finished: received all from server.
#=> {"type":"asr#nictlvcsr2","session_id":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX","status":"recog-finished","response":[{"result":"たのです。","words":["た","の","です"],"determined":true,"time":4440}]}

For lid（mimi LID）

By default, language identification runs against Japanese, English, Chinese, and Korean (lang=ja|en|zh|ko). Use --lid-options to change the target languages.

python3 lid_stream_file.py token.txt audio.raw
#=>{"session_id":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX","status":"recog-in-progress","type":"lid#ja|en|zh|ko","response":[{"lang":"ja","score":0.998649},{"lang":"ko","score":0.00091},{"lang":"zh","score":0.00025},{"lang":"en","score":0.00019}]}
#=>{"session_id":"XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX","status":"recog-finished","type":"lid#ja|en|zh|ko","response":[{"lang":"ja","score":0.998649},{"lang":"ko","score":0.00091},{"lang":"zh","score":0.00025},{"lang":"en","score":0.00019}]}

For srs（mimi SRS）

--speaker-group-id is required. You must create a speaker group, register speakers, and train with their audio in advance. See SRS Training API for details.

python3 srs_stream_file.py token.txt audio.raw --speaker-group-id YOUR_GROUP_ID
#=> {"status": "recog-finished", "type": "srs#identification#YOUR_IDENTIFICATION_ID", "response": {"speaker": [{"confidence": 0.99988, "speaker_id": "YOUR_SPEAKER_ID"}, {"confidence": 0.00012, "speaker_id": ""}]}, "session_id": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"}

Speech Recognition with a microphone ( microphone.py )

When executing "microphone.py", "start recording" is displayed in your console. Once it's displayed, the system is ready to record. Press Ctrl+C immediately after your last word to get complete recognition results.

python3 microphone.py access_token_file [options]

python3 microphone.py token.txt -f v2
#=> {"type":"asr#nictlvcsr2","session_id":"5636ae8f-414f-4feb-aa44-256231306a43","status":"recog-finished","response":[{"result":"あらゆる現 実をすべて自分のほうへねじ曲げたのだ。","words":["あらゆる","現実","を","すべて","自分","の","ほう","へ","ねじ曲げ","た","の","だ"],"determined":true,"time":0}]}
#=> recog-finished: received all from server.

• Note:
  • Press Ctrl+C immediately after your last word. Without it, the server waits for recog-break and times out after 30 seconds, discarding the trailing words.
  • When input overflow is displayed, add the blocksize parameter to sd.InputStream() function and adjust it.
    e.g. sd.InputStream(channels=1, dtype="int16", blocksize=4096, callback=callback)

Machine Translation ( tra.py )

python3 tra.py access_token_file input_lang input_text output_lang

ex.
python3 tra.py token.txt ja "こんにちは" en
#=> ['Hello.']

Speech Synthesis ( tts.py )

python3 tts.py access_token_file input_lang input_text output_filename

ex.
python3 tts.py token.txt ja "こんにちは" out.wav

Further Reading

For details on each service, including supported languages and options, see the mimi API documentation and mimi API Reference.