Keyword spotting on Android

Detect spoken keywords in real time on Android with an Edge Impulse audio model and the phone’s microphone.

This tutorial runs a keyword spotting model on Android. You can use the same approach for wake words, voice commands, and other audio events.

What you’ll build

An Android app that:

• Captures audio from the microphone
• Classifies the audio continuously on-device
• Displays the detected keyword and its confidence score

Prerequisites

• A trained keyword spotting model
• Android Studio with the NDK and CMake installed
• An Android device with a microphone (a USB camera with a built-in microphone also works)

1. Clone the repository

git clone https://github.com/edgeimpulse/example-android-inferencing.git
cd example-android-inferencing/example_kws

2. Download TensorFlow Lite libraries

cd app/src/main/cpp/tflite

# macOS/Linux
sh download_tflite_libs.sh

# Windows
download_tflite_libs.bat

3. Export your audio model

1. In Edge Impulse Studio, go to Deployment.
2. Select Android (C++ library).
3. Enable EON Compiler.
4. Click Build and download the .zip file.

4. Integrate the model

Extract the downloaded .zip file and copy its contents, except CMakeLists.txt, into example_kws/app/src/main/cpp/. The folder then looks like this:

app/src/main/cpp/
├── edge-impulse-sdk/
├── model-parameters/
├── tflite-model/
├── native-lib.cpp
└── CMakeLists.txt   # keep the existing file

5. Check the audio permissions

AndroidManifest.xml already declares the microphone permission:

<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-feature android:name="android.hardware.microphone" />

6. Build and run

1. Open the project in Android Studio.
2. Select Build -> Make Project.
3. Connect your Android device.
4. Run the app and grant the microphone permission when prompted.

Speak one of your keywords. The app shows the detected keyword and its confidence score.

How it works

Audio capture

// MainActivity.kt
private fun startAudioRecording() {
    val bufferSize = AudioRecord.getMinBufferSize(
        SAMPLE_RATE,
        AudioFormat.CHANNEL_IN_MONO,
        AudioFormat.ENCODING_PCM_16BIT
    )
    
    audioRecord = AudioRecord(
        MediaRecorder.AudioSource.MIC,
        SAMPLE_RATE,
        AudioFormat.CHANNEL_IN_MONO,
        AudioFormat.ENCODING_PCM_16BIT,
        bufferSize
    )
    
    audioRecord.startRecording()
    
    // Read audio in background thread
    Thread {
        val buffer = ShortArray(bufferSize)
        while (isRecording) {
            val read = audioRecord.read(buffer, 0, buffer.size)
            if (read > 0) {
                processAudioBuffer(buffer, read)
            }
        }
    }.start()
}

Ring buffer for continuous inference

private val audioRingBuffer = RingBuffer(16000) // 1 second at 16kHz

private fun processAudioBuffer(buffer: ShortArray, size: Int) {
    // Add to ring buffer
    audioRingBuffer.write(buffer, size)
    
    // Run inference when buffer is full
    if (audioRingBuffer.isFull()) {
        val features = audioRingBuffer.read()
        val result = runInference(features)
        
        runOnUiThread {
            updateUI(result)
        }
    }
}

Native inference

// native-lib.cpp
extern "C" JNIEXPORT jobject JNICALL
Java_com_example_kws_EIClassifierAudio_run(
    JNIEnv* env, jobject, jshortArray audioData) {
    
    // Convert audio to float features
    jshort* audio = env->GetShortArrayElements(audioData, nullptr);
    int length = env->GetArrayLength(audioData);
    
    std::vector<float> features(length);
    for (int i = 0; i < length; i++) {
        features[i] = (float)audio[i] / 32768.0f;
    }
    
    // Create signal
    signal_t signal;
    signal.total_length = features.size();
    signal.get_data = &get_feature_data;
    
    // Run classifier
    ei_impulse_result_t result = {0};
    run_classifier(&signal, &result, false);
    
    // Return result object
    return createResultObject(env, result);
}

Result display

private fun updateUI(result: ClassificationResult) {
    // Find highest confidence prediction
    val topResult = result.classification.maxByOrNull { it.score } ?: return
    
    if (topResult.score > CONFIDENCE_THRESHOLD) {
        // Show detected keyword
        keywordTextView.text = topResult.label
        confidenceTextView.text = "${(topResult.score * 100).toInt()}%"
        
        // Highlight detection
        detectionIndicator.setBackgroundColor(Color.GREEN)
        
        // Trigger action (optional)
        onKeywordDetected(topResult.label)
    } else {
        keywordTextView.text = "Listening..."
        detectionIndicator.setBackgroundColor(Color.GRAY)
    }
}

Troubleshooting

No microphone permission prompt

Confirm <uses-permission android:name="android.permission.RECORD_AUDIO" /> is in AndroidManifest.xml and that you call ActivityCompat.requestPermissions on first launch. If the system dialog never appears, grant the permission under Settings -> Apps -> your app -> Permissions.

Inference runs but no keyword is ever detected

• Verify the sample rate in MainActivity.kt matches the value reported by ei_classifier_frequency() (16 kHz for most KWS models).
• Confirm EI_CLASSIFIER_SLICE_SIZE matches the slice size you read from the microphone before each run_classifier_continuous call.
• Make sure your model was exported with EON Compiler enabled and that the labels in ei_classifier_inferencing_categories match what you trained on.

Build fails with undefined symbol: aligned_alloc

The bundled TensorFlow Lite runtime uses aligned_alloc, which needs Android API 28 or later. Set minSdk to 28 or higher in app/build.gradle.kts.

App crashes on startup with UnsatisfiedLinkError

The native library failed to load. Confirm ndk { abiFilters += "arm64-v8a" } is set in app/build.gradle.kts and that download_tflite_libs.sh placed .a files in app/src/main/cpp/tflite/android64/.

Next steps

• Camera inference on Android
• QNN hardware acceleration
• Android series overview

Resources

• GitHub: example_kws
• Keyword spotting tutorial