TMIC: App Inventor Extension for the Deployment of Image Classification Models Exported from Teachable Machine

[cgvw]

TMIC: App Inventor Extension for the Deployment of Image Classification Models Exported from Teachable Machine
Version: 1.0
Released: August, 25 2022
Tested Android 9, 12, 13, companion and compiled

TMIC is an App Inventor extension for the deployment of ML models for image classification developed with Google Teachable Machine in educational settings.
Google Teachable Machine is an intuitive visual tool that provides workflow-oriented support for the development of ML models for image classification. Aiming at the usage of models developed with Google Teachable Machine, the extension TMIC enables the deployment of the trained models as part of App Inventor, one of the most popular block-based programming environments for teaching computing in K-12.

The extension TMIC enables the import of ML models created with Google Teachable Machine and exported as Tensorflow.js uploaded on Google Cloud. It allows running the trained models on Google Cloud into the App Inventor app, and capturing images with the device’s rear-facing camera.

The extension is based on the PIC extension, which enables the deployment of ML models created with the PIC web application. It includes the following properties and blocks:

The extension was created with the App Inventor extension framework and is available under the BSD 3 license and it is included with a LICENSE file.

Available material:

• TMIC extension .aix
• TMIC extension source code
• Example app for the classification of recycling trash (English version .aia and .apk/Portuguese version .aia and .apk)
• Online tutorial explaining the use of the extension (in Portuguese)

The extension is being developed by the initiative Computação na Escola of the Department of Informatics and Statistics of the Federal University of Santa Catarina/Brazil as part of a research effort aiming at introducing AI education in K-12.

F. Pereira de Oliveira, C. Gresse von Wangenheim, J. C. R. Hauck. TMIC: App Inventor Extension for the Deployment of Image Classification Models Exported from Teachable Machine. arXiv, 2022.

We would love to see the apps you create with this extension! Share the apps using the hashtag #cneufsc

[ChrisWard]

What a beautiful extension. I can foresee many people using it for Appathon 2023.

[Taifun]

thank you for your contribution
one minor issue concerning the naming conventions...

Naming Conventions for Extension Developers

If you are developing an extension, then please follow the naming conventions, which is UpperCamelCase for the package name as well as for property, method and event names (i.e. the first letter should be a capital letter) and lowerCamelCase for parameter names, no underscores . Example [naming] What does UpperCamelCase mean? UpperCamelCase (part of CamelCase ) is a naming convention in which a name is formed of multiple words that are joined together as a single word with the first l…

the correct name for the URL_Model property would be UrlModel, please adjust it accordingly, thank you

Taifun

[cgvw]

Hello, thanks for pointing this out. We will be working on the correction of this problem.

[cgvw]

@cgvw

TMIC: App Inventor Extension for the Deployment of Image Classification Models Exported from Teachable Machine
Version: 2.0 (correcting naming conventions)
Released: August, 30 2022
Tested Android 9, 12, 13, companion and compiled

Available material:

• TMIC extension .aix
• TMIC extension source code
• Example app for the classification of recycling trash (English version .aia and .apk/Portuguese version .aia and .apk)

• Online tutorial explaining the use of the extension (in Portuguese)

[garridoferfer]

Hello, I have tried the extension in android emulators, studio and genymotion, and it doesn't work. It is important in the classroom to be able to use these emulators with our students, due to the variety of mobiles they have.
It would be very useful if the extension were adapted.
Thank you for your work.

[SteveJG]

Welcome @garridoferfer .

@garridoferfer

it doesn't work

Did you load the provided aia Fernando and use it with Companion and a real device?

Of course the example does not work in an emulator. It does work on a cell phone.
To use the example that is provided with the extension you must use a camera. I am not aware of any emulator that has camera hardware.

[garridoferfer]

Hello, in Companion and in android emulators it doesn't work, but it does in real device.
Thanks.

[Ravi_Rituraj]

Is there a way to use mobile front camera to capture image ?

[cgvw]

At the moment with this version of the extension there is no possibility to capture images with the mobile front camera.

[Ravi_Rituraj]

ok , I will look forward for this update in future.

[mag]

hello, is it possible to use the extension with the device offline?

[cgvw]

Hello, this version of the extension does not allow an offline use.

[ISRAEL_ISAYA_MARCO]

Hello..
I'm the beginner, I would need someone who can kindly help me step by step on developing my app​:pray:.
How can I add videos of the identified object using this version of extension.

[cgvw]

Hello
This version of the extension does not enable the addition of videos of the identified object.

[ISRAEL_ISAYA_MARCO]

Ok thanks

[Memo]

Hello, is there a way to classify an image with this extension?

[cgvw]

Hello, currently its only possible to classify pictures taken with the camera. More information: Computação na Escola

[CSeducator_Mindchamp]

Hi
I am using the above model in my app but it doesnt seem to work . Can someone please help. The model works in the browser. app is attached below, I am using Android 13 google phone, MIT AI Companion recently downloaded.
Plant_disease_detection.aia (469.4 KB)

[cgvw]

Hi, there seems to be a problem with the .tm model you exported, as in the app it keeps running, never returning any classification result. Did you export the Tensorflow.js version?