Kickstarter is now live!Learn more →
Developer Documentation

EIO Developer Kit

Complete documentation for developing applications on the EIO Developer Kit platform. Built on Android 9 AOSP, providing direct access to hardware APIs including camera, microphone, speaker, and physical controls.

System Overview

The EIO Developer Kit is built on Android 9 AOSP. This documentation covers the essential hardware APIs for building immersive AR applications:

⌨️Event Key Listening

Handle physical button inputs (scroll, confirm, back, power)

📸Camera APIs

Capture and process visual data using CameraX or Camera2

🎤Microphone APIs

Record audio using MediaRecorder

🔊Speaker APIs

Audio output and text-to-speech functionality

Quick Start Guide

Follow these steps to set up your development environment and create your first EIO Developer Kit application.

Prerequisites

Before you begin, ensure you have the following installed on your development machine:

RequirementVersionNotes
Android StudioHedgehog (2023.1.1) or laterOfficial IDE for Android development
JDK11 or 17Required for Android Gradle Plugin
Android SDKAPI Level 28+ (Android 9.0)Target platform for Developer Kit
Gradle8.0+Build automation tool
USB DriverLatestFor device debugging via ADB

Step 1: Install Android Studio

Download and install Android Studio from the official website. During installation, make sure to include:

  • Android SDK
  • Android SDK Platform-Tools
  • Android Virtual Device (optional, for emulator testing)

Step 2: Configure SDK

Open Android Studio and navigate to Settings → Languages & Frameworks → Android SDK. Install the following components:

SDK ConfigurationPlain Text
1SDK Platforms:
2 ✓ Android 9.0 (Pie) - API Level 28
3 ✓ Android 10.0 (Q) - API Level 29 (recommended for testing)
4
5SDK Tools:
6 ✓ Android SDK Build-Tools (latest)
7 ✓ Android SDK Platform-Tools
8 ✓ Android Emulator (optional)
9 ✓ Google USB Driver (Windows only)

Step 3: Create a New Project

Create a new Android project with the following configuration:

  1. Open Android Studio and select New Project
  2. Choose Empty Views Activity template
  3. Set the following options:
    • Name: MyEIOApp
    • Package name: com.example.myeioapp
    • Language: Kotlin or Java
    • Minimum SDK: API 28 (Android 9.0 Pie)
  4. Click Finish to create the project

Step 4: Configure build.gradle

Open your app-level build.gradle and configure it for the Developer Kit:

app/build.gradleGradle
1plugins {
2 id 'com.android.application'
3 id 'kotlin-android' // or remove for Java-only project
4}
5
6android {
7 namespace 'com.example.myeioapp'
8 compileSdk 33
9
10 defaultConfig {
11 applicationId "com.example.myeioapp"
12 minSdk 28 // Android 9.0 - Developer Kit minimum
13 targetSdk 33
14 versionCode 1
15 versionName "1.0"
16 }
17
18 buildTypes {
19 release {
20 minifyEnabled false
21 proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
22 }
23 }
24
25 compileOptions {
26 sourceCompatibility JavaVersion.VERSION_11
27 targetCompatibility JavaVersion.VERSION_11
28 }
29
30 buildFeatures {
31 viewBinding true // Recommended for cleaner UI code
32 }
33}
34
35dependencies {
36 implementation 'androidx.core:core-ktx:1.12.0'
37 implementation 'androidx.appcompat:appcompat:1.6.1'
38 implementation 'com.google.android.material:material:1.11.0'
39 implementation 'androidx.constraintlayout:constraintlayout:2.1.4'
40
41 // Add these as needed for specific features:
42 // CameraX (Section 2)
43 // def camerax_version = "1.3.0"
44 // implementation "androidx.camera:camera-core:$camerax_version"
45 // implementation "androidx.camera:camera-camera2:$camerax_version"
46 // implementation "androidx.camera:camera-lifecycle:$camerax_version"
47 // implementation "androidx.camera:camera-view:$camerax_version"
48}

Step 5: Connect Your Device

Connect the EIO Developer Kit to your computer via USB:

  1. Enable Developer Options on the device:
    • Go to Settings → About Device
    • Tap Build Number 7 times
  2. Enable USB Debugging:
    • Go to Settings → Developer Options
    • Enable USB Debugging
  3. Connect via USB and accept the debugging prompt on the device
  4. Verify connection in terminal:
TerminalShell
1# Check connected devices
2adb devices
3
4# Expected output:
5# List of devices attached
6# XXXXXXXX device

Step 6: Run Your App

With your device connected:

  1. Select your device from the device dropdown in Android Studio
  2. Click the Run button (▶) or press Shift + F10
  3. Wait for the app to build and install on the device
  4. Your app should launch automatically on the Developer Kit
NoteIf you encounter connection issues, try running adb kill-server followed by adb start-server to reset the ADB connection.

Next Steps

Now that your development environment is set up, you can start integrating the hardware APIs. We recommend following this learning path:

1️⃣

Key Events

Start with physical button handling - it's the simplest API and essential for navigation.

2️⃣

Camera APIs

Add camera preview for AR applications - the core visual input for smart glasses.

3️⃣

Microphone APIs

Enable voice input and audio recording for voice commands and dictation.

4️⃣

Speaker APIs

Add audio feedback and text-to-speech for a complete interactive experience.

Build Your First AR Application

Let's build a simple but complete AR application in about 15 minutes. This hands-on tutorial will create a "Smart Viewer" app that displays a camera preview and responds to button presses with audio feedback.

What we'll build:

Camera PreviewButton ControlsVoice FeedbackStatus Display

Step 1: Create the Project

Create a new Android project following the Quick Start Guide above, then add CameraX dependencies to your app/build.gradle:

app/build.gradle (add to dependencies)Gradle
1dependencies {
2 // ... existing dependencies ...
3
4 // CameraX
5 def camerax_version = "1.3.0"
6 implementation "androidx.camera:camera-core:$camerax_version"
7 implementation "androidx.camera:camera-camera2:$camerax_version"
8 implementation "androidx.camera:camera-lifecycle:$camerax_version"
9 implementation "androidx.camera:camera-view:$camerax_version"
10}

Step 2: Add Permissions

Add camera permission to AndroidManifest.xml:

AndroidManifest.xmlXML
1<manifest xmlns:android="http://schemas.android.com/apk/res/android">
2
3 <!-- Add this line before <application> -->
4 <uses-permission android:name="android.permission.CAMERA" />
5 <uses-feature android:name="android.hardware.camera.any" />
6
7 <application ...>
8 <!-- your activity -->
9 </application>
10</manifest>

Step 3: Create the Layout

Replace the content of activity_main.xml:

res/layout/activity_main.xmlXML
1<?xml version="1.0" encoding="utf-8"?>
2<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
3 android:layout_width="match_parent"
4 android:layout_height="match_parent"
5 android:background="#000000">
6
7 <!-- Camera Preview (fullscreen) -->
8 <androidx.camera.view.PreviewView
9 android:id="@+id/viewFinder"
10 android:layout_width="match_parent"
11 android:layout_height="match_parent" />
12
13 <!-- Status Overlay -->
14 <LinearLayout
15 android:layout_width="match_parent"
16 android:layout_height="wrap_content"
17 android:layout_gravity="bottom"
18 android:orientation="vertical"
19 android:padding="24dp"
20 android:background="#80000000">
21
22 <TextView
23 android:id="@+id/statusText"
24 android:layout_width="wrap_content"
25 android:layout_height="wrap_content"
26 android:text="Smart Viewer Ready"
27 android:textColor="#FFFFFF"
28 android:textSize="24sp"
29 android:textStyle="bold" />
30
31 <TextView
32 android:id="@+id/hintText"
33 android:layout_width="wrap_content"
34 android:layout_height="wrap_content"
35 android:layout_marginTop="8dp"
36 android:text="Use scroll wheel to navigate • Press OK to capture"
37 android:textColor="#AAAAAA"
38 android:textSize="14sp" />
39
40 </LinearLayout>
41
42</FrameLayout>

Step 4: Write the MainActivity

Replace the content of MainActivity.java:

MainActivity.javaJava
1package com.example.smartviewer;
2
3import android.Manifest;
4import android.content.pm.PackageManager;
5import android.os.Bundle;
6import android.speech.tts.TextToSpeech;
7import android.view.KeyEvent;
8import android.widget.TextView;
9import android.widget.Toast;
10
11import androidx.appcompat.app.AppCompatActivity;
12import androidx.camera.core.CameraSelector;
13import androidx.camera.core.Preview;
14import androidx.camera.lifecycle.ProcessCameraProvider;
15import androidx.camera.view.PreviewView;
16import androidx.core.app.ActivityCompat;
17import androidx.core.content.ContextCompat;
18
19import com.google.common.util.concurrent.ListenableFuture;
20
21import java.util.Locale;
22import java.util.concurrent.ExecutionException;
23
24public class MainActivity extends AppCompatActivity {
25
26 // === UI Components ===
27 private PreviewView viewFinder;
28 private TextView statusText;
29 private TextView hintText;
30
31 // === Features ===
32 private TextToSpeech tts;
33 private int zoomLevel = 1; // 1x, 2x, 3x zoom simulation
34
35 // === Constants ===
36 private static final int REQUEST_CODE_PERMISSIONS = 10;
37 private static final int KEY_OK = 352; // Confirm button
38 private static final int KEY_LEFT = 105; // Scroll backward
39 private static final int KEY_RIGHT = 106; // Scroll forward
40
41 @Override
42 protected void onCreate(Bundle savedInstanceState) {
43 super.onCreate(savedInstanceState);
44 setContentView(R.layout.activity_main);
45
46 // Find views
47 viewFinder = findViewById(R.id.viewFinder);
48 statusText = findViewById(R.id.statusText);
49 hintText = findViewById(R.id.hintText);
50
51 // Initialize Text-to-Speech
52 tts = new TextToSpeech(this, status -> {
53 if (status == TextToSpeech.SUCCESS) {
54 tts.setLanguage(Locale.US);
55 speak("Smart Viewer ready");
56 }
57 });
58
59 // Check camera permission
60 if (allPermissionsGranted()) {
61 startCamera();
62 } else {
63 ActivityCompat.requestPermissions(this,
64 new String[]{Manifest.permission.CAMERA},
65 REQUEST_CODE_PERMISSIONS);
66 }
67 }
68
69 // =============================================
70 // CAMERA SETUP
71 // =============================================
72 private void startCamera() {
73 ListenableFuture<ProcessCameraProvider> cameraProviderFuture =
74 ProcessCameraProvider.getInstance(this);
75
76 cameraProviderFuture.addListener(() -> {
77 try {
78 ProcessCameraProvider cameraProvider = cameraProviderFuture.get();
79
80 // Build Preview
81 Preview preview = new Preview.Builder().build();
82 preview.setSurfaceProvider(viewFinder.getSurfaceProvider());
83
84 // Select back camera
85 CameraSelector cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA;
86
87 // Bind to lifecycle
88 cameraProvider.unbindAll();
89 cameraProvider.bindToLifecycle(this, cameraSelector, preview);
90
91 updateStatus("Camera active", "Scroll: zoom • OK: capture");
92
93 } catch (ExecutionException | InterruptedException e) {
94 Toast.makeText(this, "Camera init failed", Toast.LENGTH_SHORT).show();
95 }
96 }, ContextCompat.getMainExecutor(this));
97 }
98
99 // =============================================
100 // KEY EVENT HANDLING - This is the core AR interaction!
101 // =============================================
102 @Override
103 public boolean dispatchKeyEvent(KeyEvent event) {
104 // Only handle key down events
105 if (event.getAction() != KeyEvent.ACTION_DOWN) {
106 return super.dispatchKeyEvent(event);
107 }
108
109 switch (event.getKeyCode()) {
110 case KEY_LEFT:
111 case KeyEvent.KEYCODE_DPAD_LEFT:
112 // Zoom out
113 zoomLevel = Math.max(1, zoomLevel - 1);
114 updateStatus("Zoom: " + zoomLevel + "x", "");
115 speak("Zoom " + zoomLevel + " x");
116 return true;
117
118 case KEY_RIGHT:
119 case KeyEvent.KEYCODE_DPAD_RIGHT:
120 // Zoom in
121 zoomLevel = Math.min(3, zoomLevel + 1);
122 updateStatus("Zoom: " + zoomLevel + "x", "");
123 speak("Zoom " + zoomLevel + " x");
124 return true;
125
126 case KEY_OK:
127 case KeyEvent.KEYCODE_DPAD_CENTER:
128 // Capture action
129 updateStatus("Captured!", "Image saved");
130 speak("Photo captured");
131 // In a real app, you would capture an image here
132 return true;
133
134 case KeyEvent.KEYCODE_BACK:
135 case 158: // KEY_BACK
136 speak("Goodbye");
137 finish();
138 return true;
139 }
140
141 return super.dispatchKeyEvent(event);
142 }
143
144 // =============================================
145 // HELPER METHODS
146 // =============================================
147 private void updateStatus(String status, String hint) {
148 runOnUiThread(() -> {
149 statusText.setText(status);
150 if (!hint.isEmpty()) {
151 hintText.setText(hint);
152 }
153 });
154 }
155
156 private void speak(String text) {
157 if (tts != null) {
158 tts.speak(text, TextToSpeech.QUEUE_FLUSH, null, "utterance");
159 }
160 }
161
162 private boolean allPermissionsGranted() {
163 return ContextCompat.checkSelfPermission(this, Manifest.permission.CAMERA)
164 == PackageManager.PERMISSION_GRANTED;
165 }
166
167 @Override
168 public void onRequestPermissionsResult(int requestCode, String[] permissions, int[] grantResults) {
169 super.onRequestPermissionsResult(requestCode, permissions, grantResults);
170 if (requestCode == REQUEST_CODE_PERMISSIONS) {
171 if (allPermissionsGranted()) {
172 startCamera();
173 } else {
174 Toast.makeText(this, "Camera permission required", Toast.LENGTH_SHORT).show();
175 finish();
176 }
177 }
178 }
179
180 @Override
181 protected void onDestroy() {
182 super.onDestroy();
183 if (tts != null) {
184 tts.stop();
185 tts.shutdown();
186 }
187 }
188}

Step 5: Build and Run

  1. Sync Gradle - Click "Sync Now" when prompted after adding dependencies
  2. Connect Device - Plug in your EIO Developer Kit via USB
  3. Run - Press the Run button (▶) in Android Studio
  4. Test - Try the controls:
    • Scroll wheel left/right → Zoom level changes with voice feedback
    • Press OK button → "Photo captured" message with voice
    • Press Back → App exits with "Goodbye"
🎉

Congratulations!

You've built your first AR application! This simple app demonstrates the core interaction pattern for AR glasses: visual input (camera), physical controls (buttons), and audio output (TTS). Explore the API sections below to add more features.

What's Next?

Now that you have a working app, you can extend it with:

Image Capture

Add ImageCapture use case to save photos

See Camera APIs →

Voice Commands

Record audio for voice-activated features

See Microphone APIs →

More Controls

Handle F1, F2, and other hardware keys

See Key Events →

1. Event Key Listening

The EIO Developer Kit features physical controls that map to standard Android key events. These can be captured in your Activity to provide intuitive navigation and interaction.

What you'll learn:

  • Map physical buttons to Android key codes
  • Override dispatchKeyEvent() to capture button presses
  • Provide visual feedback with animations
  • Support multiple device types and key mappings

Key Definitions

The following table shows the primary hardware key mappings:

Physical EventMap EventHex CodeDecimal Code
Scroll ForwardKEY_RIGHT0x6a106
Scroll BackwardKEY_LEFT0x69105
ConfirmKEY_OK0x160352
ReturnKEY_BACK0x9e158
PowerPOWER0x74116

Extended Key Events

Additional keys available on compatible AR glasses (e.g., C2000 series):

Key CodePhysical ButtonDescription
KEYCODE_F1SOS KeyEmergency or quick action button
KEYCODE_F2Record KeyRed dot / recording toggle
KEYCODE_DPAD_LEFTWheel CCWScroll wheel counterclockwise
KEYCODE_DPAD_RIGHTWheel CWScroll wheel clockwise
KEYCODE_DPAD_CENTERWheel PressScroll wheel pressed
KEYCODE_BACKBack KeyReturn / navigate back
NoteThe physical scroll wheel generates KEY_LEFT and KEY_RIGHT events. Use these for list navigation, volume control, or any scrollable content in your application.

Implementation Example

Override dispatchKeyEvent() in your Activity to capture and handle key events:

MainActivity.javaJava
1package com.example.keyeventdemo;
2
3import android.os.Bundle;
4import android.view.KeyEvent;
5import android.widget.Toast;
6import androidx.appcompat.app.AppCompatActivity;
7
8public class MainActivity extends AppCompatActivity {
9
10 // Define Key Constants
11 public static final int KEY_RIGHT = 106; // 0x6a - Scroll forward
12 public static final int KEY_LEFT = 105; // 0x69 - Scroll backward
13 public static final int KEY_OK = 352; // 0x160 - Confirm button
14 public static final int KEY_BACK = 158; // 0x9e - Return button
15 public static final int KEY_POWER = 116; // 0x74 - Power button
16
17 @Override
18 protected void onCreate(Bundle savedInstanceState) {
19 super.onCreate(savedInstanceState);
20 setContentView(R.layout.activity_main);
21 }
22
23 @Override
24 public boolean dispatchKeyEvent(KeyEvent event) {
25 int keyCode = event.getKeyCode();
26 int action = event.getAction();
27
28 if (action == KeyEvent.ACTION_DOWN) {
29 switch (keyCode) {
30 case KEY_LEFT:
31 showToast("Scroll backward");
32 // Handle backward navigation
33 return true;
34 case KEY_RIGHT:
35 showToast("Scroll forward");
36 // Handle forward navigation
37 return true;
38 case KEY_OK:
39 showToast("Confirm pressed");
40 // Handle selection/confirmation
41 return true;
42 case KEY_BACK:
43 showToast("Return pressed");
44 super.onBackPressed();
45 return true;
46 }
47 }
48
49 // Pass unhandled events to the system
50 return super.dispatchKeyEvent(event);
51 }
52
53 private void showToast(String message) {
54 Toast.makeText(this, message, Toast.LENGTH_SHORT).show();
55 }
56}

UI Animation Feedback

Enhance user experience with visual feedback animations when physical buttons are pressed:

KeyAnimationHelper.javaJava
1// Add scale animation feedback for button press
2public void animateButtonPress(View targetView, String message) {
3 targetView.post(() -> {
4 // Update text if it's a TextView/Button
5 if (targetView instanceof TextView) {
6 ((TextView) targetView).setText(message);
7 }
8
9 // Create scale animation sequence
10 ObjectAnimator scaleXUp = ObjectAnimator.ofFloat(
11 targetView, "scaleX", 1f, 1.2f);
12 ObjectAnimator scaleYUp = ObjectAnimator.ofFloat(
13 targetView, "scaleY", 1f, 1.2f);
14 ObjectAnimator scaleXDown = ObjectAnimator.ofFloat(
15 targetView, "scaleX", 1.2f, 1f);
16 ObjectAnimator scaleYDown = ObjectAnimator.ofFloat(
17 targetView, "scaleY", 1.2f, 1f);
18
19 // Play animations in sequence
20 AnimatorSet animatorSet = new AnimatorSet();
21 animatorSet.playSequentially(
22 scaleXUp, scaleYUp, scaleXDown, scaleYDown);
23 animatorSet.setDuration(100); // 100ms per animation
24 animatorSet.start();
25 });
26}
27
28// Usage in onKeyDown
29@Override
30public boolean onKeyDown(int keyCode, KeyEvent event) {
31 String message;
32
33 switch (keyCode) {
34 case KeyEvent.KEYCODE_F1:
35 message = "SOS key pressed";
36 break;
37 case KeyEvent.KEYCODE_F2:
38 message = "Record key pressed";
39 break;
40 case KeyEvent.KEYCODE_DPAD_CENTER:
41 message = "Wheel pressed";
42 break;
43 default:
44 return super.onKeyDown(keyCode, event);
45 }
46
47 animateButtonPress(feedbackButton, message);
48 return true;
49}

2. Camera APIs

With button handling in place, let's add visual capabilities. The camera is the primary sensor for AR applications, enabling scene understanding, object detection, and visual assistance features.

Access the device camera using the CameraX API for preview, capture, and image analysis. CameraX provides a consistent API across Android devices with lifecycle-aware components. For advanced control, Camera2 API is also available.

What you'll learn:

  • Configure CameraX dependencies and permissions
  • Display live camera preview in your app
  • Understand Camera2 core components for advanced use cases
  • Handle lifecycle and resolution switching
  • Troubleshoot common camera issues

Camera2 Core Components

When using the Camera2 API for low-level camera control, you'll work with these key components:

ComponentTypeDescription
TextureViewUI ComponentDisplays camera preview, receives image data via SurfaceTexture
CameraDeviceCamera HandleRepresents the physical camera, used for creating capture requests
CameraCaptureSessionSession ManagerManages capture request execution, routes output to target surfaces
CaptureRequest.BuilderRequest BuilderConfigures preview settings (autofocus, exposure, etc.)
CameraManagerSystem ServiceLists available cameras, queries characteristics, opens camera
HandlerThreadBackground ThreadExecutes time-consuming camera operations off the UI thread

Camera2 Initialization Flow

The Camera2 initialization follows this sequence:

  1. UI Setup - Initialize TextureView and set SurfaceTextureListener
  2. Permission Check - Verify CAMERA permission before opening
  3. Get Camera Info - Use CameraManager to get camera ID and characteristics
  4. Get Supported Resolutions - Query StreamConfigurationMap for output sizes
  5. Open Camera - Call cameraManager.openCamera with state callback
  6. Start Preview - Create capture session and start repeating request

Dependencies

Add the following CameraX dependencies to your build.gradle:

app/build.gradleGradle
1plugins {
2 id 'com.android.application'
3 id 'kotlin-android'
4}
5
6android {
7 compileSdk 33
8 defaultConfig {
9 applicationId "com.example.cameraximageview"
10 minSdk 21
11 targetSdk 33
12 versionCode 1
13 versionName "1.0"
14 }
15 buildFeatures {
16 viewBinding true
17 }
18}
19
20dependencies {
21 implementation "androidx.core:core-ktx:1.12.0"
22 implementation "androidx.appcompat:appcompat:1.6.1"
23 implementation "com.google.android.material:material:1.9.0"
24 implementation "androidx.constraintlayout:constraintlayout:2.1.4"
25
26 // CameraX dependencies
27 def camerax_version = "1.3.0"
28 implementation "androidx.camera:camera-core:$camerax_version"
29 implementation "androidx.camera:camera-camera2:$camerax_version"
30 implementation "androidx.camera:camera-lifecycle:$camerax_version"
31 implementation "androidx.camera:camera-view:$camerax_version"
32}

Permissions

Declare camera permission in your AndroidManifest.xml:

AndroidManifest.xmlXML
1<manifest xmlns:android="http://schemas.android.com/apk/res/android"
2 package="com.example.cameraximageview">
3
4 <uses-permission android:name="android.permission.CAMERA"/>
5 <uses-feature android:name="android.hardware.camera.any" />
6
7 <application
8 android:allowBackup="true"
9 android:label="CameraXImageView"
10 android:theme="@style/Theme.AppCompat.Light.DarkActionBar">
11 <activity android:name=".MainActivity"
12 android:exported="true">
13 <intent-filter>
14 <action android:name="android.intent.action.MAIN" />
15 <category android:name="android.intent.category.LAUNCHER"/>
16 </intent-filter>
17 </activity>
18 </application>
19</manifest>

Layout

activity_main.xmlXML
1<?xml version="1.0" encoding="utf-8"?>
2<androidx.constraintlayout.widget.ConstraintLayout
3 xmlns:android="http://schemas.android.com/apk/res/android"
4 xmlns:app="http://schemas.android.com/apk/res-auto"
5 android:layout_width="match_parent"
6 android:layout_height="match_parent">
7
8 <androidx.camera.view.PreviewView
9 android:id="@+id/previewView"
10 android:layout_width="0dp"
11 android:layout_height="0dp"
12 app:layout_constraintTop_toTopOf="parent"
13 app:layout_constraintBottom_toTopOf="@+id/toggleButton"
14 app:layout_constraintStart_toStartOf="parent"
15 app:layout_constraintEnd_toEndOf="parent"/>
16
17 <Button
18 android:id="@+id/toggleButton"
19 android:layout_width="wrap_content"
20 android:layout_height="wrap_content"
21 android:text="Close Camera"
22 app:layout_constraintTop_toBottomOf="@id/previewView"
23 app:layout_constraintBottom_toBottomOf="parent"
24 app:layout_constraintStart_toStartOf="parent"
25 app:layout_constraintEnd_toEndOf="parent"/>
26</androidx.constraintlayout.widget.ConstraintLayout>

Camera Implementation

The following Kotlin implementation demonstrates camera preview with toggle functionality:

MainActivity.ktKotlin
1package com.example.cameraximageview
2
3import android.Manifest
4import android.content.pm.PackageManager
5import android.os.Bundle
6import android.view.View
7import androidx.appcompat.app.AppCompatActivity
8import androidx.camera.core.CameraSelector
9import androidx.camera.core.Preview
10import androidx.camera.lifecycle.ProcessCameraProvider
11import androidx.core.app.ActivityCompat
12import androidx.core.content.ContextCompat
13import com.example.cameraximageview.databinding.ActivityMainBinding
14
15class MainActivity : AppCompatActivity() {
16 private lateinit var binding: ActivityMainBinding
17 private var cameraProvider: ProcessCameraProvider? = null
18 private var isCameraOn = true
19
20 companion object {
21 private const val PERMISSION_REQUEST_CAMERA = 10
22 }
23
24 override fun onCreate(savedInstanceState: Bundle?) {
25 super.onCreate(savedInstanceState)
26 binding = ActivityMainBinding.inflate(layoutInflater)
27 setContentView(binding.root)
28
29 // Check and request camera permission
30 if (allPermissionsGranted()) {
31 startCamera()
32 } else {
33 ActivityCompat.requestPermissions(
34 this,
35 arrayOf(Manifest.permission.CAMERA),
36 PERMISSION_REQUEST_CAMERA
37 )
38 }
39
40 // Toggle button listener
41 binding.toggleButton.setOnClickListener {
42 isCameraOn = !isCameraOn
43 if (isCameraOn) {
44 binding.previewView.visibility = View.VISIBLE
45 startCamera()
46 binding.toggleButton.text = "Close Camera"
47 } else {
48 binding.previewView.visibility = View.INVISIBLE
49 stopCamera()
50 binding.toggleButton.text = "Open Camera"
51 }
52 }
53 }
54
55 private fun allPermissionsGranted(): Boolean {
56 return ContextCompat.checkSelfPermission(
57 this, Manifest.permission.CAMERA
58 ) == PackageManager.PERMISSION_GRANTED
59 }
60
61 private fun startCamera() {
62 val cameraProviderFuture = ProcessCameraProvider.getInstance(this)
63 cameraProviderFuture.addListener({
64 cameraProvider = cameraProviderFuture.get()
65
66 // Build preview use case
67 val preview = Preview.Builder().build().also {
68 it.setSurfaceProvider(binding.previewView.surfaceProvider)
69 }
70
71 // Select back camera
72 val cameraSelector = CameraSelector.DEFAULT_BACK_CAMERA
73
74 // Bind use cases to lifecycle
75 cameraProvider?.unbindAll()
76 cameraProvider?.bindToLifecycle(this, cameraSelector, preview)
77 }, ContextCompat.getMainExecutor(this))
78 }
79
80 private fun stopCamera() {
81 cameraProvider?.unbindAll()
82 }
83
84 override fun onRequestPermissionsResult(
85 requestCode: Int,
86 permissions: Array<out String>,
87 grantResults: IntArray
88 ) {
89 super.onRequestPermissionsResult(requestCode, permissions, grantResults)
90 if (requestCode == PERMISSION_REQUEST_CAMERA) {
91 if (allPermissionsGranted()) {
92 startCamera()
93 } else {
94 finish() // Permission denied, close app
95 }
96 }
97 }
98}
NoteFor image capture and analysis, add additional CameraX use cases such asImageCapture andImageAnalysisto your lifecycle binding.

Camera Lifecycle Management

Proper lifecycle management is critical for camera applications:

Lifecycle MethodCamera Actions
onCreateInitialize UI, set listeners, configure camera manager
onResumeStart background thread, check TextureView availability, open camera
onPauseClose camera and session, stop background thread, release resources
onDestroyFinal cleanup of any remaining resources

Resolution Switching

To switch camera resolution dynamically:

ResolutionSwitch.javaJava
1// Get supported resolutions from camera characteristics
2StreamConfigurationMap map = characteristics.get(
3 CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
4Size[] supportedSizes = map.getOutputSizes(SurfaceTexture.class);
5
6// Cycle through resolutions
7private int currentSizeIndex = 0;
8
9public void switchResolution() {
10 currentSizeIndex = (currentSizeIndex + 1) % supportedSizes.length;
11 restartCamera(); // Close and reopen with new resolution
12}
13
14private void restartCamera() {
15 closeCamera();
16 openCamera(); // Will use supportedSizes[currentSizeIndex]
17}

Common Camera Troubleshooting

IssueSolution
Camera won't openCheck permission declaration and runtime grant; look for CameraAccessException
Preview issuesVerify resolution and buffer size; ensure Surface is added to capture request
Resolution switch failsEnsure supportedSizes is populated; call restartCamera properly
App crashesAvoid heavy operations on UI thread; release resources during lifecycle transitions

3. Microphone APIs

Now that you have visual input from the camera, let's add audio input. Voice commands and audio recording enable hands-free interaction, which is essential for AR glasses where users may not have access to a touchscreen.

Record audio using Android's MediaRecorder API. This allows you to capture voice input, ambient audio, or any sound for processing and playback.

What you'll learn:

  • Request and handle audio recording permissions
  • Configure MediaRecorder for different quality levels
  • Implement recording, playback, and pause/resume controls
  • Build reusable audio module interfaces
  • Integrate with physical button controls

Permissions

AndroidManifest.xmlXML
1<?xml version="1.0" encoding="utf-8"?>
2<manifest xmlns:android="http://schemas.android.com/apk/res/android"
3 package="com.example.microphonetest">
4
5 <uses-permission android:name="android.permission.RECORD_AUDIO" />
6 <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
7
8 <application
9 android:allowBackup="true"
10 android:icon="@mipmap/ic_launcher"
11 android:label="@string/app_name"
12 android:theme="@style/AppTheme">
13 <activity android:name=".MainActivity"
14 android:exported="true">
15 <intent-filter>
16 <action android:name="android.intent.action.MAIN" />
17 <category android:name="android.intent.category.LAUNCHER" />
18 </intent-filter>
19 </activity>
20 </application>
21</manifest>

Layout

activity_main.xmlXML
1<?xml version="1.0" encoding="utf-8"?>
2<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
3 android:layout_width="match_parent"
4 android:layout_height="match_parent"
5 android:orientation="vertical"
6 android:padding="16dp"
7 android:gravity="center">
8
9 <Button
10 android:id="@+id/btnRecord"
11 android:layout_width="match_parent"
12 android:layout_height="wrap_content"
13 android:text="Start Recording"
14 android:textSize="18sp"
15 android:layout_marginBottom="16dp"/>
16
17 <Button
18 android:id="@+id/btnStop"
19 android:layout_width="match_parent"
20 android:layout_height="wrap_content"
21 android:text="Stop Recording"
22 android:textSize="18sp"
23 android:layout_marginBottom="16dp"/>
24
25 <Button
26 android:id="@+id/btnPlay"
27 android:layout_width="match_parent"
28 android:layout_height="wrap_content"
29 android:text="Play Recording"
30 android:textSize="18sp"/>
31</LinearLayout>

Recording Implementation

The following implementation provides complete recording, stopping, and playback functionality:

MainActivity.javaJava
1package com.example.microphonetest;
2
3import androidx.annotation.NonNull;
4import androidx.appcompat.app.AppCompatActivity;
5import androidx.core.app.ActivityCompat;
6import androidx.core.content.ContextCompat;
7
8import android.Manifest;
9import android.content.pm.PackageManager;
10import android.media.MediaPlayer;
11import android.media.MediaRecorder;
12import android.os.Bundle;
13import android.view.View;
14import android.widget.Button;
15import android.widget.Toast;
16
17import java.io.IOException;
18
19public class MainActivity extends AppCompatActivity {
20
21 private static final int REQUEST_RECORD_AUDIO_PERMISSION = 200;
22 private String fileName = null;
23
24 private Button recordButton, stopButton, playButton;
25 private MediaRecorder recorder = null;
26 private MediaPlayer player = null;
27
28 private boolean permissionToRecordAccepted = false;
29 private String[] permissions = {
30 Manifest.permission.RECORD_AUDIO,
31 Manifest.permission.WRITE_EXTERNAL_STORAGE
32 };
33
34 @Override
35 protected void onCreate(Bundle savedInstanceState) {
36 super.onCreate(savedInstanceState);
37 setContentView(R.layout.activity_main);
38
39 // Set output file path
40 fileName = getExternalCacheDir().getAbsolutePath();
41 fileName += "/audiorecordtest.3gp";
42
43 // Initialize UI
44 recordButton = findViewById(R.id.btnRecord);
45 stopButton = findViewById(R.id.btnStop);
46 playButton = findViewById(R.id.btnPlay);
47
48 // Initial button states
49 stopButton.setEnabled(false);
50 playButton.setEnabled(false);
51
52 // Request permissions
53 ActivityCompat.requestPermissions(this, permissions,
54 REQUEST_RECORD_AUDIO_PERMISSION);
55
56 // Button listeners
57 recordButton.setOnClickListener(v -> startRecording());
58 stopButton.setOnClickListener(v -> stopRecording());
59 playButton.setOnClickListener(v -> startPlaying());
60 }
61
62 @Override
63 public void onRequestPermissionsResult(int requestCode,
64 @NonNull String[] permissions, @NonNull int[] grantResults) {
65 super.onRequestPermissionsResult(requestCode, permissions, grantResults);
66 if (requestCode == REQUEST_RECORD_AUDIO_PERMISSION) {
67 permissionToRecordAccepted =
68 grantResults[0] == PackageManager.PERMISSION_GRANTED;
69 }
70 if (!permissionToRecordAccepted) finish();
71 }
72
73 private void startRecording() {
74 recorder = new MediaRecorder();
75 recorder.setAudioSource(MediaRecorder.AudioSource.MIC);
76 recorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP);
77 recorder.setOutputFile(fileName);
78 recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);
79
80 try {
81 recorder.prepare();
82 } catch (IOException e) {
83 e.printStackTrace();
84 return;
85 }
86
87 recorder.start();
88
89 // Update UI
90 recordButton.setEnabled(false);
91 stopButton.setEnabled(true);
92 playButton.setEnabled(false);
93 Toast.makeText(this, "Recording started", Toast.LENGTH_SHORT).show();
94 }
95
96 private void stopRecording() {
97 if (recorder != null) {
98 recorder.stop();
99 recorder.release();
100 recorder = null;
101
102 // Update UI
103 recordButton.setEnabled(true);
104 stopButton.setEnabled(false);
105 playButton.setEnabled(true);
106 Toast.makeText(this, "Recording saved", Toast.LENGTH_SHORT).show();
107 }
108 }
109
110 private void startPlaying() {
111 player = new MediaPlayer();
112 try {
113 player.setDataSource(fileName);
114 player.prepare();
115 player.start();
116
117 // Update UI during playback
118 recordButton.setEnabled(false);
119 stopButton.setEnabled(false);
120 playButton.setEnabled(false);
121
122 // Re-enable buttons when playback completes
123 player.setOnCompletionListener(mp -> {
124 releasePlayer();
125 recordButton.setEnabled(true);
126 playButton.setEnabled(true);
127 Toast.makeText(this, "Playback finished",
128 Toast.LENGTH_SHORT).show();
129 });
130
131 Toast.makeText(this, "Playing...", Toast.LENGTH_SHORT).show();
132 } catch (IOException e) {
133 e.printStackTrace();
134 Toast.makeText(this, "Playback failed: " + e.getMessage(),
135 Toast.LENGTH_SHORT).show();
136 }
137 }
138
139 private void releasePlayer() {
140 if (player != null) {
141 player.release();
142 player = null;
143 }
144 }
145
146 @Override
147 protected void onStop() {
148 super.onStop();
149 if (recorder != null) {
150 recorder.release();
151 recorder = null;
152 }
153 releasePlayer();
154 }
155}
NoteFor better audio quality, consider using AudioRecord with PCM encoding for raw audio access, or AAC encoder for higher quality compressed audio.

Advanced Audio Configuration

For higher quality recording, use these MediaRecorder settings:

AdvancedRecording.javaJava
1private void startHighQualityRecording() {
2 recorder = new MediaRecorder();
3
4 // Use VOICE_RECOGNITION for AR glasses microphone
5 // Alternative: MediaRecorder.AudioSource.MIC for standard mic
6 recorder.setAudioSource(MediaRecorder.AudioSource.VOICE_RECOGNITION);
7
8 // Use MPEG_4 format with AAC encoder for better quality
9 recorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
10 recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);
11
12 // Higher bitrate for better quality (128kbps)
13 recorder.setAudioEncodingBitRate(128000);
14
15 // 44.1kHz sample rate (CD quality)
16 recorder.setAudioSamplingRate(44100);
17
18 // Set output file
19 String filePath = getExternalFilesDir(Environment.DIRECTORY_MUSIC)
20 .getAbsolutePath() + "/recording.mp4";
21 recorder.setOutputFile(filePath);
22
23 try {
24 recorder.prepare();
25 recorder.start();
26 } catch (IOException e) {
27 e.printStackTrace();
28 }
29}
30
31// Auto-stop recording after specified duration
32private void startTimedRecording(int durationMs) {
33 startHighQualityRecording();
34 handler.postDelayed(() -> stopRecording(), durationMs);
35}

Secondary Development Interfaces

Recommended interfaces for building reusable audio modules:

FeatureSuggested Interface
Recording controlstartRecording(int durationMs) - configurable duration
Playback controlstartPlayback(), pausePlayback(), resumePlayback()
State listenerOnAudioStateListener - notifies recording/playback state changes
File managementgetLastRecordingFilePath(), deleteRecordingFile()
AudioStateListener.javaJava
1// State callback interface for audio module
2public interface OnAudioStateListener {
3 void onRecordingStart();
4 void onRecordingStop();
5 void onPlaybackStart();
6 void onPlaybackComplete();
7}
8
9// Pause/Resume support
10public void togglePlayback() {
11 if (mediaPlayer != null) {
12 if (mediaPlayer.isPlaying()) {
13 mediaPlayer.pause();
14 listener.onPlaybackPause();
15 } else {
16 mediaPlayer.start();
17 listener.onPlaybackResume();
18 }
19 }
20}

Remote Control Integration

Support for AR glasses physical controls and remote devices:

RemoteControlSupport.javaJava
1@Override
2public boolean dispatchKeyEvent(KeyEvent event) {
3 if (event.getAction() != KeyEvent.ACTION_DOWN) {
4 return super.dispatchKeyEvent(event);
5 }
6
7 switch (event.getKeyCode()) {
8 case KeyEvent.KEYCODE_DPAD_LEFT:
9 case KeyEvent.KEYCODE_DPAD_RIGHT:
10 // Switch focus between Record/Play buttons
11 switchButtonFocus();
12 return true;
13
14 case KeyEvent.KEYCODE_DPAD_CENTER:
15 // Trigger focused button action
16 executeFocusedAction();
17 return true;
18 }
19 return super.dispatchKeyEvent(event);
20}

Important Notes

  • Always check recording permission before initializing MediaRecorder
  • Ensure recording file exists before playback to avoid exceptions
  • Test compatibility when using external microphones (USB audio devices)
  • Lower encoding parameters for low-end devices that don't support high bitrates
  • Recording files saved to getExternalFilesDir() are private and deleted on uninstall

4. Speaker APIs

To complete the audio loop, let's add output capabilities. Audio feedback is crucial for AR applications where users need confirmation of actions, navigation guidance, or text reading without looking at a screen.

Output audio through the device speaker using Android's AudioManager and TextToSpeech APIs. This enables voice feedback, notifications, and text-to-speech functionality.

What you'll learn:

  • Initialize and configure TextToSpeech engine
  • Control audio routing and speaker settings
  • Customize speech parameters (pitch, speed, language)
  • Handle TTS lifecycle and resource cleanup

Layout

activity_main.xmlXML
1<?xml version="1.0" encoding="utf-8"?>
2<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
3 android:layout_width="match_parent"
4 android:layout_height="match_parent"
5 android:orientation="vertical"
6 android:padding="16dp"
7 android:gravity="center_horizontal"
8 android:background="#f5f5f5">
9
10 <ImageView
11 android:layout_width="120dp"
12 android:layout_height="120dp"
13 android:src="@android:drawable/ic_btn_speak_now"
14 android:layout_gravity="center"
15 android:layout_marginTop="30dp"
16 android:layout_marginBottom="20dp"/>
17
18 <EditText
19 android:id="@+id/textInput"
20 android:layout_width="match_parent"
21 android:layout_height="wrap_content"
22 android:hint="Enter text to speak"
23 android:minLines="5"
24 android:maxLines="8"
25 android:padding="16dp"
26 android:background="@android:color/white"
27 android:layout_marginBottom="24dp"
28 android:textSize="16sp"/>
29
30 <Button
31 android:id="@+id/speakButton"
32 android:layout_width="match_parent"
33 android:layout_height="wrap_content"
34 android:text="Speak"
35 android:textSize="18sp"
36 android:padding="18dp"
37 android:layout_marginBottom="30dp"/>
38
39 <TextView
40 android:id="@+id/statusText"
41 android:layout_width="wrap_content"
42 android:layout_height="wrap_content"
43 android:text="Status: Ready"
44 android:textSize="16sp"
45 android:textStyle="bold"/>
46</LinearLayout>

Text-to-Speech Implementation

This implementation provides text-to-speech with speaker control:

MainActivity.javaJava
1package com.example.speakertest;
2
3import android.content.Context;
4import android.media.AudioManager;
5import android.os.Bundle;
6import android.speech.tts.TextToSpeech;
7import android.view.View;
8import android.widget.Button;
9import android.widget.EditText;
10import android.widget.TextView;
11import android.widget.Toast;
12
13import androidx.appcompat.app.AppCompatActivity;
14
15import java.util.Locale;
16
17public class MainActivity extends AppCompatActivity {
18
19 private AudioManager audioManager;
20 private Button speakButton;
21 private EditText textInput;
22 private TextView statusText;
23 private TextToSpeech textToSpeech;
24 private boolean isSpeaking = false;
25
26 @Override
27 protected void onCreate(Bundle savedInstanceState) {
28 super.onCreate(savedInstanceState);
29 setContentView(R.layout.activity_main);
30
31 // Initialize AudioManager for speaker control
32 audioManager = (AudioManager) getSystemService(Context.AUDIO_SERVICE);
33
34 // Initialize UI components
35 speakButton = findViewById(R.id.speakButton);
36 textInput = findViewById(R.id.textInput);
37 statusText = findViewById(R.id.statusText);
38
39 // Set default text
40 textInput.setText("Hello! This is a test of the text-to-speech system.");
41
42 // Button click listener
43 speakButton.setOnClickListener(v -> toggleSpeech());
44
45 // Initialize TextToSpeech engine
46 textToSpeech = new TextToSpeech(this, status -> {
47 if (status == TextToSpeech.SUCCESS) {
48 // Set language to US English
49 int result = textToSpeech.setLanguage(Locale.US);
50 if (result == TextToSpeech.LANG_MISSING_DATA ||
51 result == TextToSpeech.LANG_NOT_SUPPORTED) {
52 showToast("Language not supported");
53 } else {
54 // Configure speech parameters
55 textToSpeech.setPitch(1.0f); // Normal pitch
56 textToSpeech.setSpeechRate(1.0f); // Normal speed
57 }
58 } else {
59 showToast("TTS initialization failed");
60 }
61 });
62 }
63
64 private void toggleSpeech() {
65 String text = textInput.getText().toString().trim();
66 if (text.isEmpty()) {
67 showToast("Please enter text to speak");
68 return;
69 }
70
71 try {
72 if (isSpeaking) {
73 stopSpeaking();
74 speakButton.setText("Speak");
75 statusText.setText("Status: Stopped");
76 } else {
77 startSpeaking(text);
78 speakButton.setText("Stop");
79 statusText.setText("Status: Speaking...");
80 }
81 isSpeaking = !isSpeaking;
82 } catch (Exception e) {
83 e.printStackTrace();
84 showToast("Error: " + e.getMessage());
85 }
86 }
87
88 private void startSpeaking(String text) {
89 // Enable speaker output
90 audioManager.setMode(AudioManager.MODE_NORMAL);
91 audioManager.setSpeakerphoneOn(true);
92
93 // Speak the text
94 if (textToSpeech != null) {
95 textToSpeech.stop();
96 textToSpeech.speak(text, TextToSpeech.QUEUE_FLUSH, null, "tts1");
97 }
98 }
99
100 private void stopSpeaking() {
101 if (textToSpeech != null) {
102 textToSpeech.stop();
103 }
104 audioManager.setSpeakerphoneOn(false);
105 }
106
107 private void showToast(String message) {
108 Toast.makeText(this, message, Toast.LENGTH_SHORT).show();
109 }
110
111 @Override
112 protected void onDestroy() {
113 super.onDestroy();
114 // Clean up TTS resources
115 if (textToSpeech != null) {
116 textToSpeech.stop();
117 textToSpeech.shutdown();
118 }
119 }
120}
NoteYou can customize the speech output by adjusting setPitch() (range: 0.5 - 2.0) and setSpeechRate() (range: 0.5 - 2.0). For multi-language support, check available locales with textToSpeech.getAvailableLanguages().

Audio Output Configuration

Key AudioManager methods for speaker control:

MethodDescription
setMode(MODE_NORMAL)Normal audio mode for media playback
setMode(MODE_IN_COMMUNICATION)Optimized for voice communication
setSpeakerphoneOn(true)Route audio to the loudspeaker
setStreamVolume()Adjust volume for specific audio streams
adjustVolume()Raise or lower volume incrementally

5. Putting It All Together

Now that you understand each API individually, let's combine them into a complete application. This example demonstrates a simple AR assistant that uses all four hardware APIs.

Application Features:

  • Camera preview for visual context
  • Physical button navigation between modes
  • Voice recording for commands
  • Text-to-speech feedback

Project Structure

Project StructurePlain Text
1app/
2├── src/main/
3│ ├── java/com/example/eioassistant/
4│ │ ├── MainActivity.java # Main entry point
5│ │ ├── CameraHelper.java # Camera initialization & control
6│ │ ├── AudioHelper.java # Recording & playback
7│ │ └── TTSHelper.java # Text-to-speech wrapper
8│ ├── res/
9│ │ ├── layout/
10│ │ │ └── activity_main.xml # Main layout
11│ │ └── values/
12│ │ └── strings.xml # String resources
13│ └── AndroidManifest.xml # Permissions & config
14└── build.gradle # Dependencies

AndroidManifest.xml

First, declare all required permissions:

AndroidManifest.xmlXML
1<?xml version="1.0" encoding="utf-8"?>
2<manifest xmlns:android="http://schemas.android.com/apk/res/android"
3 package="com.example.eioassistant">
4
5 <!-- Camera permission -->
6 <uses-permission android:name="android.permission.CAMERA" />
7 <uses-feature android:name="android.hardware.camera.any" />
8
9 <!-- Audio permissions -->
10 <uses-permission android:name="android.permission.RECORD_AUDIO" />
11 <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
12
13 <application
14 android:allowBackup="true"
15 android:label="@string/app_name"
16 android:theme="@style/Theme.AppCompat.Light.DarkActionBar">
17 <activity
18 android:name=".MainActivity"
19 android:exported="true"
20 android:screenOrientation="landscape">
21 <intent-filter>
22 <action android:name="android.intent.action.MAIN" />
23 <category android:name="android.intent.category.LAUNCHER" />
24 </intent-filter>
25 </activity>
26 </application>
27</manifest>

Main Activity Integration

Here's how to integrate all APIs in a single Activity:

MainActivity.javaJava
1package com.example.eioassistant;
2
3import android.Manifest;
4import android.content.pm.PackageManager;
5import android.os.Bundle;
6import android.speech.tts.TextToSpeech;
7import android.view.KeyEvent;
8import android.view.View;
9import android.widget.Button;
10import android.widget.TextView;
11import androidx.appcompat.app.AppCompatActivity;
12import androidx.camera.view.PreviewView;
13import androidx.core.app.ActivityCompat;
14import java.util.Locale;
15
16public class MainActivity extends AppCompatActivity {
17
18 // UI Components
19 private PreviewView cameraPreview;
20 private TextView statusText;
21 private Button actionButton;
22
23 // Helpers
24 private CameraHelper cameraHelper;
25 private AudioHelper audioHelper;
26 private TextToSpeech tts;
27
28 // State
29 private enum Mode { IDLE, CAMERA, RECORDING, SPEAKING }
30 private Mode currentMode = Mode.IDLE;
31
32 private static final int PERMISSION_REQUEST = 100;
33 private String[] permissions = {
34 Manifest.permission.CAMERA,
35 Manifest.permission.RECORD_AUDIO
36 };
37
38 @Override
39 protected void onCreate(Bundle savedInstanceState) {
40 super.onCreate(savedInstanceState);
41 setContentView(R.layout.activity_main);
42
43 // Initialize UI
44 cameraPreview = findViewById(R.id.cameraPreview);
45 statusText = findViewById(R.id.statusText);
46 actionButton = findViewById(R.id.actionButton);
47
48 // Request permissions
49 if (!allPermissionsGranted()) {
50 ActivityCompat.requestPermissions(this, permissions, PERMISSION_REQUEST);
51 } else {
52 initializeComponents();
53 }
54
55 // Button click handler
56 actionButton.setOnClickListener(v -> handleActionButton());
57 }
58
59 private void initializeComponents() {
60 // Initialize Camera
61 cameraHelper = new CameraHelper(this, cameraPreview);
62
63 // Initialize Audio Recorder
64 audioHelper = new AudioHelper(this);
65
66 // Initialize TTS
67 tts = new TextToSpeech(this, status -> {
68 if (status == TextToSpeech.SUCCESS) {
69 tts.setLanguage(Locale.US);
70 speak("EIO Assistant ready");
71 }
72 });
73
74 updateUI();
75 }
76
77 // ========== KEY EVENT HANDLING ==========
78 @Override
79 public boolean dispatchKeyEvent(KeyEvent event) {
80 if (event.getAction() != KeyEvent.ACTION_DOWN) {
81 return super.dispatchKeyEvent(event);
82 }
83
84 switch (event.getKeyCode()) {
85 case KeyEvent.KEYCODE_DPAD_LEFT:
86 // Previous mode
87 cycleMode(-1);
88 return true;
89
90 case KeyEvent.KEYCODE_DPAD_RIGHT:
91 // Next mode
92 cycleMode(1);
93 return true;
94
95 case KeyEvent.KEYCODE_DPAD_CENTER:
96 case 352: // KEY_OK
97 // Execute current mode action
98 handleActionButton();
99 return true;
100
101 case KeyEvent.KEYCODE_BACK:
102 case 158: // KEY_BACK
103 // Stop current action and return to idle
104 stopCurrentAction();
105 currentMode = Mode.IDLE;
106 updateUI();
107 return true;
108 }
109
110 return super.dispatchKeyEvent(event);
111 }
112
113 private void cycleMode(int direction) {
114 Mode[] modes = Mode.values();
115 int newIndex = (currentMode.ordinal() + direction + modes.length) % modes.length;
116 currentMode = modes[newIndex];
117 speak("Mode: " + currentMode.name());
118 updateUI();
119 }
120
121 // ========== ACTION HANDLING ==========
122 private void handleActionButton() {
123 switch (currentMode) {
124 case CAMERA:
125 if (cameraHelper.isActive()) {
126 cameraHelper.stop();
127 speak("Camera stopped");
128 } else {
129 cameraHelper.start();
130 speak("Camera started");
131 }
132 break;
133
134 case RECORDING:
135 if (audioHelper.isRecording()) {
136 audioHelper.stopRecording();
137 speak("Recording saved");
138 } else {
139 audioHelper.startRecording();
140 speak("Recording started");
141 }
142 break;
143
144 case SPEAKING:
145 if (audioHelper.hasRecording()) {
146 audioHelper.playRecording();
147 speak("Playing recording");
148 } else {
149 speak("No recording available");
150 }
151 break;
152
153 default:
154 speak("Select a mode first");
155 }
156 updateUI();
157 }
158
159 private void stopCurrentAction() {
160 cameraHelper.stop();
161 audioHelper.stopRecording();
162 audioHelper.stopPlayback();
163 }
164
165 // ========== TTS HELPER ==========
166 private void speak(String text) {
167 if (tts != null) {
168 tts.speak(text, TextToSpeech.QUEUE_FLUSH, null, "utterance");
169 }
170 }
171
172 // ========== UI UPDATE ==========
173 private void updateUI() {
174 String status = "Mode: " + currentMode.name();
175 statusText.setText(status);
176
177 switch (currentMode) {
178 case CAMERA:
179 actionButton.setText(cameraHelper.isActive() ? "Stop Camera" : "Start Camera");
180 cameraPreview.setVisibility(View.VISIBLE);
181 break;
182 case RECORDING:
183 actionButton.setText(audioHelper.isRecording() ? "Stop Recording" : "Start Recording");
184 cameraPreview.setVisibility(View.GONE);
185 break;
186 case SPEAKING:
187 actionButton.setText("Play Recording");
188 cameraPreview.setVisibility(View.GONE);
189 break;
190 default:
191 actionButton.setText("Select Mode");
192 cameraPreview.setVisibility(View.GONE);
193 }
194 }
195
196 // ========== PERMISSIONS ==========
197 private boolean allPermissionsGranted() {
198 for (String permission : permissions) {
199 if (ActivityCompat.checkSelfPermission(this, permission)
200 != PackageManager.PERMISSION_GRANTED) {
201 return false;
202 }
203 }
204 return true;
205 }
206
207 @Override
208 public void onRequestPermissionsResult(int requestCode,
209 String[] permissions, int[] grantResults) {
210 super.onRequestPermissionsResult(requestCode, permissions, grantResults);
211 if (requestCode == PERMISSION_REQUEST && allPermissionsGranted()) {
212 initializeComponents();
213 }
214 }
215
216 // ========== LIFECYCLE ==========
217 @Override
218 protected void onDestroy() {
219 super.onDestroy();
220 if (tts != null) {
221 tts.stop();
222 tts.shutdown();
223 }
224 stopCurrentAction();
225 }
226}

Summary Checklist

Before deploying your app to the EIO Developer Kit, verify these items:

All permissions declared in AndroidManifest.xml
Runtime permission requests implemented
Key event handling for physical buttons
Proper lifecycle management (onResume, onPause, onDestroy)
Resource cleanup for camera, audio, and TTS
Audio feedback for user actions
NoteThis example provides a foundation for building AR applications. Extend it with features like image recognition, voice command processing, or network connectivity based on your application requirements.

Additional Resources

Explore more resources to help you build amazing applications on the EIO Developer Kit platform.

EIO logo
EIO