pocketsphinx run on Cortex-A55 with OS Linux

Golton Kwok — Thu, 15 Aug 2024 02:59:04 -0000

I run the same demo on x64 Ubuntu with intel cpu i7-13700H and on ARM Cortex-A55, with the same dictionary, language model and acoustic model But the recognition time running on ARM Cortex-A55 (about 10 seconds )is slower than on x64 Ubuntu with intel cpu i7-13700H (within 1 seconds). So how to optimize for ARM Cortex-A55 platform as I want to run pocketsphinx on embedded Linux.

How to solve the problem of identifying too accurately?

g lk — Mon, 29 Jan 2024 08:18:40 -0000

entry file:

    from pocketsphinx import Endpointer, Decoder, set_loglevel, Config, get_model_path
    import os
    import sounddevice
    import asyncio
    from util.string import get_absolute_path

    class Porcupine():
        model_path =  get_absolute_path("offline_wakeup\\cmusphinx-zh-cn-5.2_back\\zh_cn.cd_cont_5000")
        lm = get_absolute_path('offline_wakeup\\Sphinx_Keyword_zh_cn\\', '1225.lm')
        dic =  get_absolute_path("offline_wakeup\\Sphinx_Keyword_zh_cn\\", "1225.dic")

        print("model_path", model_path)
        print("lm", lm)
        print("dic", dic)

        sampleRate = 16000

        def __init__(self, on_status_change=None, config=None):
            self.on_status_change = on_status_change

            # 是否在听
            self.is_listening = False

            self.ep = Endpointer(vad_mode=3)

            self.block_size = self.ep.frame_bytes // 2

            self.initialize()

        def initialize(self):
            """
            """
            """
                pocketsphinx.Config Doc: https://pocketsphinx.readthedocs.io/en/latest/pocketsphinx.html#pocketsphinx.Config
                Config Doc: https://pocketsphinx.readthedocs.io/en/latest/config_params.html#Config
            """

            try:
                config = Config(
                    hmm=self.model_path, 
                    lm=self.lm, 
                    dict=self.dic, 
                    samprate=self.sampleRate, 
                    # remove_noise=True
                )
            except Exception as e:
              print('Config error', e)

            try:
                self.stream = sounddevice.RawInputStream(
                    samplerate=self.sampleRate,
                    blocksize=self.block_size,
                    dtype="int16",
                    channels=1,
                )
            except Exception as e:
              print('RawInputStream error', e)

            self.decoder = Decoder(config)

        def _status_change(self, data=None):
            self.on_status_change and asyncio.run(self.on_status_change(data))

        def start(self,):
            with self.stream:
                print("请说...")
                self.is_listening = True
                self._status_change({ 
                    "t": "is_listening",
                })
                try:
                    while True:
                        frame, _ = self.stream.read(self.block_size)
                        prev_in_speech = self.ep.in_speech
                        speech = self.ep.process(frame)
                        if speech is not None:
                            if not prev_in_speech:
                                print("开始时间：", self.ep.speech_start)
                                self.decoder.start_utt()
                            self.decoder.process_raw(speech)

                            # 获取当前的识别假设。
                            hyp = self.decoder.hyp()

                            # 如果有识别结果
                            if hyp is not None:
                                print("PARTIAL RESULT: ", hyp.hypstr)

                            # 如果不在说话了
                            if not self.ep.in_speech:
                                print("结束时间：", self.ep.speech_end)
                                self.decoder.end_utt()
                                result = self.decoder.hyp().hypstr
                                print("识别结果：", result)
                                print("评分", self.decoder.hyp().score)
                                if any(value in result for value in ["小薇", "小微"]):
                                    print("😀")
                                    self._status_change({ 
                                        "t": "keyword",
                                        "val": result
                                    })
                except Exception as e:
                    self._status_change({"t": "error", "val": f"endpointer error: {str(e)}" })
                    print('出现异常', e)

        def stop(self,):
            if self.stream:
                self.stream.abort()


    if __name__ == "__main__":
        porcupine_instance = Porcupine()
        porcupine_instance.start()

        # while True:
        #     pass

After the program starts, I don't speak, but it recognizes the other keywords I set. At the same time, if I accurately say the keywords I set, it can also trigger keywords. However, I don't want other confusing keywords to be easily triggered. What should I do? Thank you to everyone.

Pocketsphinx tutorial refers to utilities that didn't install

Korey F Schulz — Tue, 09 Jan 2024 05:27:19 -0000

Hi. I am updating the model for an application that I am building but I can't make progress in the tutorial because none of the utilities that are referenced exist on my machine. The missing utilities include sphinx_fe, pocketsphinx_mdef_convert, bw, mllr_solve, map_adapt, mk_s2sendump, and pocketsphinx_continuous.

I installed pocketsphinx and sphinxbase but the problem still exists. I'm running Ubuntu. Can somebody help me?

Speech recognition for english is pretty bad

HACHICHA YOUSSEF — Sun, 07 Jan 2024 10:32:10 -0000

what do you mean by that i already checked that in order to get it up and running but clearly they didn't do a good job, like what do they mean by :collects over 20 years of the CMU research
clearly 20 years went to waste

Speech recognition for english is pretty bad

HACHICHA YOUSSEF — Sun, 07 Jan 2024 10:29:14 -0000

what do you mean by that i already used it and its not accuarate at all

Speech recognition for english is pretty bad

HACHICHA YOUSSEF — Sat, 06 Jan 2024 14:00:13 -0000

I dont know what it is but no matter what file i give it either a video or audio the transcription is not accuarate at all, even tho that the files are clear english it's really bad. do i have to train it? if yes can someone tell me how to do it in python?
that's my code bellow ↓↓↓

also its 2024 arent you planning on changing the conversation over to slack or some else better?

Speech recognition for english is pretty bad

HACHICHA YOUSSEF — Sat, 06 Jan 2024 14:00:13 -0000

also its 2024 arent you planning on changing the conversation over to slack or some else better?

Speech recognition for english is pretty bad

HACHICHA YOUSSEF — Sat, 06 Jan 2024 14:00:06 -0000

also its 2024 arent you planning on changing the conversation over to slack or some else better?

Speech Recognition for english is very bad

HACHICHA YOUSSEF — Sat, 06 Jan 2024 13:58:29 -0000

App getting crashed without any error while launching

Aswathy Mohan — Fri, 22 Dec 2023 05:17:11 -0000

Hi,
I have created a sample app for voice recognition using PocketSphinx. But while launching app , it got crashed. No exceptions and error showing in the logcat. And as per the troubleshooting , i found like crash happening with getRecognizer() method call. But I am unable to fix or find the error. I'm using android versions 8 to latest(13).Please see my complete code below

import androidx.annotation.NonNull;
import androidx.appcompat.app.AppCompatActivity;
import androidx.core.app.ActivityCompat;
import androidx.core.content.ContextCompat;

import android.Manifest;
import android.content.pm.PackageManager;
import android.os.AsyncTask;
import android.os.Bundle;
import android.os.StrictMode;
import android.util.Log;
import android.widget.TextView;
import android.widget.Toast;

import java.io.File;
import java.io.IOException;
import java.lang.ref.WeakReference;
import java.util.HashMap;
import java.util.Locale;

import edu.cmu.pocketsphinx.Assets;
import edu.cmu.pocketsphinx.BuildConfig;
import edu.cmu.pocketsphinx.Hypothesis;
import edu.cmu.pocketsphinx.RecognitionListener;
import edu.cmu.pocketsphinx.SpeechRecognizer;
import edu.cmu.pocketsphinx.SpeechRecognizerSetup;

public class MainActivity extends AppCompatActivity implements
RecognitionListener

{
private static final String KWS_SEARCH = "wakeup";
private static final String FORECAST_SEARCH = "forecast";
private static final String DIGITS_SEARCH = "digits";
private static final String PHONE_SEARCH = "phones";
private static final String MENU_SEARCH = "menu";

/* Keyword we are looking for to activate menu */
private static final String KEYPHRASE = "oh mighty computer";

/* Used to handle permission request */
private static final int PERMISSIONS_REQUEST_RECORD_AUDIO = 1;
private static final int MY_PERMISSIONS_REQUEST_READ_EXTERNAL_STORAGE =2;

private SpeechRecognizer recognizer;
private HashMap<String, Integer> captions;
@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    if (BuildConfig.DEBUG) {
        StrictMode.setThreadPolicy(new StrictMode.ThreadPolicy.Builder()
                .detectAll()
                .penaltyLog()
                .build());
        StrictMode.setVmPolicy(new StrictMode.VmPolicy.Builder()
                .detectAll()
                .penaltyLog()
                .build());
    }
   captions = new HashMap<>();
   captions.put(KWS_SEARCH, R.string.kws_caption);
   captions.put(MENU_SEARCH, R.string.menu_caption);
   captions.put(DIGITS_SEARCH, R.string.digits_caption);
   captions.put(PHONE_SEARCH, R.string.phone_caption);
   captions.put(FORECAST_SEARCH, R.string.forecast_caption);
   ((TextView) findViewById(R.id.caption_text))
           .setText("Preparing the recognizer");
    if (ContextCompat.checkSelfPermission(getApplicationContext(), Manifest.permission.READ_EXTERNAL_STORAGE)
            != PackageManager.PERMISSION_GRANTED) {
        // Request permission to read files
        ActivityCompat.requestPermissions(this,
                new String[]{Manifest.permission.READ_EXTERNAL_STORAGE},
                MY_PERMISSIONS_REQUEST_READ_EXTERNAL_STORAGE);
        return;
    }
    // Check if user has given permission to record audio
    int permissionCheck = ContextCompat.checkSelfPermission(getApplicationContext(), Manifest.permission.RECORD_AUDIO);
    if (permissionCheck != PackageManager.PERMISSION_GRANTED) {
        ActivityCompat.requestPermissions(this, new String[]{android.Manifest.permission.RECORD_AUDIO}, PERMISSIONS_REQUEST_RECORD_AUDIO);
        return;
    }
    // Recognizer initialization is a time-consuming and it involves IO,
    // so we execute it in async task
    new SetupTask(this).execute();
}

private static class SetupTask extends AsyncTask<Void, Void, Exception> {
    WeakReference<MainActivity> activityReference;
    SetupTask(MainActivity activity) {
        this.activityReference = new WeakReference<>(activity);
    }
    @Override
    protected Exception doInBackground(Void... params) {
        try {
            Assets assets = new Assets(activityReference.get());
            File assetDir = assets.syncAssets();

            activityReference.get().setupRecognizer(assetDir);
        } catch (IOException e) {
            Log.d("print IO Ecception  ",e.getMessage());
            return e;
        }
        return null;
    }
    @Override
    protected void onPostExecute(Exception result) {
        if (result != null) {
            ((TextView) activityReference.get().findViewById(R.id.caption_text))
                    .setText("Failed to init recognizer " + result);
        } else {
            activityReference.get().switchSearch(KWS_SEARCH);
        }
    }
}
@Override
public void onRequestPermissionsResult(int requestCode,
                                       @NonNull String[] permissions, @NonNull  int[] grantResults) {
    super.onRequestPermissionsResult(requestCode, permissions, grantResults);

    if (requestCode == PERMISSIONS_REQUEST_RECORD_AUDIO) {
        if (grantResults.length > 0 && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
            // Recognizer initialization is a time-consuming and it involves IO,
            // so we execute it in async task
            new SetupTask(this).execute();
        } else {
            finish();
        }
    }
}

@Override
public void onDestroy() {
    super.onDestroy();

    if (recognizer != null) {
        recognizer.cancel();
        recognizer.shutdown();
    }
}

/**
 * In partial result we get quick updates about current hypothesis. In
 * keyword spotting mode we can react here, in other modes we need to wait
 * for final result in onResult.
 */
@Override
public void onPartialResult(Hypothesis hypothesis) {
    if (hypothesis == null)
        return;

    String text = hypothesis.getHypstr();
    if (text.equals(KEYPHRASE))
        switchSearch(MENU_SEARCH);
    else if (text.equals(DIGITS_SEARCH))
        switchSearch(DIGITS_SEARCH);
    else if (text.equals(PHONE_SEARCH))
        switchSearch(PHONE_SEARCH);
    else if (text.equals(FORECAST_SEARCH))
        switchSearch(FORECAST_SEARCH);
    else
        ((TextView) findViewById(R.id.result_text)).setText(text);
}

/**
 * This callback is called when we stop the recognizer.
 */
@Override
public void onResult(Hypothesis hypothesis) {
    ((TextView) findViewById(R.id.result_text)).setText("");
    if (hypothesis != null) {
        String text = hypothesis.getHypstr();
        Toast.makeText(getApplicationContext(), text, Toast.LENGTH_SHORT).show();
    }
}

@Override
public void onBeginningOfSpeech() {
}

/**
 * We stop recognizer here to get a final result
 */
@Override
public void onEndOfSpeech() {
    if (!recognizer.getSearchName().equals(KWS_SEARCH))
        switchSearch(KWS_SEARCH);
}

private void switchSearch(String searchName) {
    recognizer.stop();

    // If we are not spotting, start listening with timeout (10000 ms or 10 seconds).
    if (searchName.equals(KWS_SEARCH))
        recognizer.startListening(searchName);
    else
        recognizer.startListening(searchName, 10000);

    String caption = getResources().getString(captions.get(searchName));
    ((TextView) findViewById(R.id.caption_text)).setText(caption);
}

private void setupRecognizer(File assetsDir) throws IOException {
    // The recognizer can be configured to perform multiple searches
    // of different kind and switch between them
    File acousticModel = new File(assetsDir, "en-us-ptm");
    File dictionary = new File(assetsDir, "cmudict-en-us.dict");
    if(acousticModel.exists()) {
        Log.d("SetupRecognizer", "Acoustic Model Path: " + acousticModel.getAbsolutePath());
    }if(dictionary.exists()) {
        Log.d("SetupRecognizer", "Dictionary Path: " + dictionary.getAbsolutePath());
    }
    try {
        recognizer = SpeechRecognizerSetup.defaultSetup()
                .setAcousticModel(acousticModel)
                .setDictionary(dictionary)
                .getRecognizer();

        if (recognizer != null) {
            Log.d("SetupRecognizer", "After initializing recognizer");
            recognizer.addListener(this);
            Log.d("SetupRecognizer", "Listener attached successfully");
        } else {
            Log.e("SetupRecognizer", "Recognizer is null after initialization");
        }


} catch (IOException e) {
        Log.e("SetupRecognizer", "Error setting up recognizer", e);
        return;
    }
 }


@Override
public void onTimeout() {
    switchSearch(KWS_SEARCH);
}

@Override
public void onError(Exception e) {
    ((TextView) findViewById(R.id.caption_text)).setText(e.getMessage());

}

}

Recent posts to Forums

pocketsphinx run on Cortex-A55 with OS Linux

How to solve the problem of identifying too accurately?

entry file:

Pocketsphinx tutorial refers to utilities that didn't install

Speech recognition for english is pretty bad

Speech recognition for english is pretty bad

Speech recognition for english is pretty bad

Speech recognition for english is pretty bad

Speech recognition for english is pretty bad

Speech Recognition for english is very bad

App getting crashed without any error while launching