Context & Role

This project focuses on designing and validating user-centered dashboards for electric buses, with an emphasis on enhancing usability, safety, and driver experience in the Indian context. By addressing the unique challenges faced by bus drivers and stakeholders, the research integrates methodologies and theoretical frameworks from human-computer interaction (HCI), cognitive psychology, and user-centered design (UCD).

Challenges

• Complex Driver Tasks: Integrating real-time monitoring of battery levels, route navigation, and managing traffic hazards.
• Varying Road Conditions: Adapting the interface to diverse traffic patterns and unpredictable road behaviors.

Actions Taken

• Developed a high-fidelity prototype through iterative design cycles: user research, wireframing, and simulator testing.
• Integrated real-time sensor data (battery, motor status) into a simplified digital dashboard.
• Conducted extensive user feedback sessions with bus drivers to refine the UI and minimize cognitive load.

Results

• Reduced driver distraction and improved key response times (e.g., faster recognition of battery warnings).
• Enhanced overall safety and user acceptance, proving the feasibility of large-scale EV bus deployment.

Lessons Learned

• Merging hardware-electronics expertise with HCI is critical for an effective design.
• Field observations and iterative feedback cycles are indispensable for real-world solutions.

Context & Role

This project evaluates a voice-based Conversational AI prototype designed to facilitate banking transactions—specifically, transferring Rs 5000 to a beneficiary named "Ravi Kumar." Developed using HTML5, CSS3, Bootstrap, and JavaScript, the prototype employs the Wizard-of-Oz technique to simulate natural language understanding. The study involved remote usability testing with 40 participants to assess the system's usability, attractiveness, and intuitiveness.

Prototype Design & Dialog Structure

• Application Flow: The system guides users through a banking transaction via a flowchart-based interface.
• Dialog Prompts: Three types of prompts are used:
  • Default Prompts: Instructions provided at each step (e.g., "Please narrate your name" vs. "Your name?").
  • Timeout Prompts: Activated when no response is detected.
  • Invalid-Input Prompts: Played when incorrect inputs are given, offering corrective guidance.
• Interface Variants: Two prototype versions were created:
  • Prototype Version A: Features a dark background, detailed default prompts, and a white orb that signals listening mode.
  • Prototype Version B: Uses a light background with minimal default prompts and similar visual cues.

Challenges

• Ensuring a natural dialogue flow while managing multi-turn conversations.
• Accurately capturing and processing domain-specific banking terminology.
• Balancing the detail of dialog prompts with visual design to maintain user engagement.
• Conducting remote usability tests using video conferencing tools and a Wizard-of-Oz setup.

Actions Taken

• Implemented a Wizard-of-Oz framework, triggering pre-recorded dialog prompts via designated keystrokes to simulate AI behavior.
• Developed two distinct interface versions to assess the impact of dialog prompt style and background design on user experience.
• Conducted a comprehensive user study with 40 participants, using a series of pre- and post-interaction questionnaires:
  • Single Ease Questionnaire (SEQ) to gauge task ease.
  • System Usability Scale (SUS) for overall usability.
  • AttrakDiff’s attractiveness scale to measure efficiency, enjoyment, and appeal.
  • INTUI questionnaire to assess intuitiveness.
• Analyzed both quantitative scores (e.g., SUS scores of 92.25 and 88.63) and qualitative user feedback to determine system performance.

Results & Findings

• Participants rated the system as highly usable, attractive, and intuitive, with SEQ and SUS scores indicating an excellent user experience.
• Interface Version A received slightly higher ratings in usability and overall satisfaction compared to Version B.
• The study demonstrates that voice-based conversational AI can be a viable interface for banking applications when backed by robust natural language understanding.

Lessons Learned

• Detailed, descriptive prompts may enhance user satisfaction, though minimal prompts can also be effective depending on design context.
• Remote usability testing using the Wizard-of-Oz method can yield valuable insights, but further testing in real-world environments (e.g., at ATMs) is recommended.
• Balancing system performance, security, and natural language processing is crucial for deploying conversational AI in sensitive domains like banking.
• The findings support further investment in conversational AI for banking, with potential extensions to enhance accessibility (e.g., for the visually impaired) and support hands-free interactions.

Context & Role

AgroAssam is a web-based speech recognition application developed to help users retrieve the latest prices of agricultural commodities and weather-related information in the Assamese language. By extracting commodity price data from the AGMARKNET website and weather data from the IMD website—both updated daily by the Government of India—the system adapts an existing phone-based voice query solution to a modern web interface.

Experimental Setup & ASR Performance

• Data Collection: Speech data was gathered in real field conditions from native Assamese speakers. This involved:
  • 30 hours of isolated commodity and district names from 885 speakers.
  • 3 hours of phonetically balanced sentences from 25 speakers.
  • Additional continuous speech recordings from 27 speakers (totaling 5658 files).
• Acoustic Modeling: Two modeling techniques were explored using the Kaldi toolkit:
  • GMM-HMM: Achieved WERs of 10.31% for commodity names and 5.16% for district names.
  • DNN-HMM: Improved WERs to 7.79% for commodities and 4.98% for district names.
• Noise Handling: The system employed Zero Frequency Filtered Signal (ZFFS) techniques to effectively separate foreground speech from background noise.

Application Design

• On accessing the application URL, users are greeted with a welcome prompt and asked to state the desired district.
• Upon successful recognition of the district, users choose between retrieving agricultural commodity prices or weather information.
• For commodity queries, after the commodity name is recognized, the system checks the district-commodity combination in the database and displays the modal price.
• For weather queries, the system retrieves current and upcoming weather details for the specified district.
• The back-end ASR modules are developed with state-of-the-art acoustic modeling approaches, ensuring accurate recognition despite channel and noise variations.

Results & Impact

• The DNN-HMM based system significantly reduced the word error rate, leading to more accurate recognition of both commodity and district names.
• The integration into a web interface demonstrates that systems designed for telephonic voice queries can be effectively adapted for web-based platforms without compromising performance.
• AgroAssam has the potential for deployment in public kiosks and digital information centers, thereby supporting farmers with timely access to essential agricultural and weather information in their native language.

Conclusion

AgroAssam successfully bridges the gap between traditional phone-based voice query systems and modern web interfaces. By leveraging advanced ASR techniques and a robust experimental setup, the project provides a reliable solution for retrieving vital information in Assamese, thus empowering the agrarian community and contributing to the digital transformation initiatives in India.

Context & Role

Developed at IIT Bhubaneswar, SIFA automates form-filling in five Indian languages (Hindi, Odia, Bengali, Assamese, Telugu). I contributed to designing the acoustic model for Odia language and integrating a custom lexicon for all the respective fields of the form.

Challenges

• Data Inconsistency: Handling varied audio quality and misaligned transcripts.
• Local Vocabulary: Incorporating domain-specific terms, such as district names and local expressions.

Actions Taken

• Automated preprocessing pipelines to standardize sampling rates and clean transcripts.
• Developed and integrated a custom lexicon into an end-to-end or CTC-based ASR model.
• Employed shallow fusion with a domain-adapted language model to enhance recognition accuracy.

Results

• Achieved a significant reduction in Word Error Rate (WER), dropping from 30% to approximately 12%.
• Improved the user experience, making form completion easier and faster for non-technical users.

Lessons Learned

• High data quality is paramount in speech tasks; even minor errors can have a large impact.
• Domain adaptation in both acoustic and language modeling dramatically improves handling of local vocabulary.