{"id":1030,"date":"2021-07-16T16:22:21","date_gmt":"2021-07-16T09:22:21","guid":{"rendered":"https:\/\/bigdolphin.com.vn\/?p=1030"},"modified":"2024-03-26T15:29:04","modified_gmt":"2024-03-26T08:29:04","slug":"playing-with-maixduino-detecting-single-digits-using-pre-trained-mnist-model","status":"publish","type":"post","link":"https:\/\/bigdolphin.com.vn\/?p=1030","title":{"rendered":"Playing with Maixduino: detecting handwritten single digits using pre-trained Mnist model"},"content":{"rendered":"\n<h3 class=\"wp-block-heading\">1.&nbsp;Prepare<\/h3>\n\n\n\n<ul><li>Maixduino&nbsp;with camera<\/li><li>MaixPy<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">2.&nbsp;Transfer Mnist model to board<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">2.1 Download pre-trained model<\/h4>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"bash\" class=\"language-bash line-numbers\">wget https:\/\/github.com\/bigdolphin\/maixduino\/raw\/main\/examples_micropython\/models\/mnist_digits.kmodel<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">2.2 Transfer to board<\/h4>\n\n\n\n<p>Connect board with MaixPy IDE then go to Tools -&gt;&nbsp;transfer file to board to upload model to board. We can also use uPyLoader to upload and manage.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"392\" height=\"245\" src=\"https:\/\/bigdolphin.com.vn\/wp-content\/uploads\/2021\/07\/image-29.png\" alt=\"\" class=\"wp-image-1031\" srcset=\"https:\/\/bigdolphin.com.vn\/wp-content\/uploads\/2021\/07\/image-29.png 392w, https:\/\/bigdolphin.com.vn\/wp-content\/uploads\/2021\/07\/image-29-300x188.png 300w\" sizes=\"(max-width: 392px) 100vw, 392px\" \/><figcaption>Transfer model to board using MaixPy IDE<\/figcaption><\/figure><\/div>\n\n\n\n<h3 class=\"wp-block-heading\">3. Writing script<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">3.1 Import modules<\/h4>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python line-numbers\">import sensor, image, time, lcd\nimport KPU as kpu\nimport gc, micropython\nfrom Maix import freq, utils\nfrom machine import reset<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">3.2 Overclock <\/h4>\n\n\n\n<p>An advantage of K210 is AI accelarator hardware called KPU,this kind of hardware gives normal microcontroller hardware (limited speed and RAM) or even CPU ability to run complex machine learning applications quickly.<br>Because of limited voltage and heat dissapation on Maixduino, when we need to run both CPU and KPU of K210, we should overclock CPU to 546 MHz maximum and KPU to 450 MHz maximum.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python line-numbers\">cpu_frq, kpu_frq=freq.get()\nprint(\"\\nCPU Frq = %d MHz\" %(cpu_frq))\nprint(\"KPU Frq = %d MHz\" %(kpu_frq))\nif cpu_frq != 546 or kpu_frq != 450:\n    print(\"Removing old frequency\u2026\")\n    os.remove(\"freq.conf\")\n    print(\"Overclocking CPU to 546 MHz and KPU to 450 MHz\u2026\")\n    # kpu frequency is pll1\/kpu_div\n    freq.set (cpu=546, pll1=450, kpu_div=1)<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">3.3 Modify heap memory size<\/h4>\n\n\n\n<p>KPU core and OMV module require much heap memory to process. We should increase not only clocks but also heap size of MicroPython.<\/p>\n\n\n\n<p><a rel=\"noreferrer noopener\" href=\"https:\/\/en.maixpy.sipeed.com\/maixpy\/en\/course\/others\/mem.html\" target=\"_blank\">In MaixPy, two types of memory management are currently used, one is GC (garbage collection) and the other is system heap memory, both of which exist at the same time.<\/a> Sipeed M1 AIOT module on Maixduino has 6 MB general purpose SRAM, the default GC uses 512KiB, and the rest is used for system heap memory management.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/bigdolphin.com.vn\/wp-content\/uploads\/2021\/07\/image-35.png\" alt=\"\" class=\"wp-image-1091\" width=\"489\" height=\"242\" srcset=\"https:\/\/bigdolphin.com.vn\/wp-content\/uploads\/2021\/07\/image-35.png 1002w, https:\/\/bigdolphin.com.vn\/wp-content\/uploads\/2021\/07\/image-35-300x149.png 300w, https:\/\/bigdolphin.com.vn\/wp-content\/uploads\/2021\/07\/image-35-768x382.png 768w\" sizes=\"(max-width: 489px) 100vw, 489px\" \/><figcaption>Memory layout of Micropython<\/figcaption><\/figure><\/div>\n\n\n\n<p>The total size of GC memory can be set. Therefore, the size of GC memory can be modified appropriately according to the specific usage, for example:<\/p>\n\n\n\n<ul><li>In order to load a larger model, you can set the GC memory setting smaller<\/li><li>If the allocation of new variables indicates insufficient memory, you can appropriately set the GC memory to be larger<\/li><li>If it is not enough, consider reducing the firmware size or optimizing the code<\/li><\/ul>\n\n\n\n<p>In this tutorial, we need to load model and run many  image processing functions, therefore, we need to decrease GC heap size down to 384KiB.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python line-numbers\">gc.enable()\ngc.collect()\ngc.threshold(gc.mem_free() \/\/ 4 + gc.mem_alloc())\nmicropython.mem_info()\nmem_heap = utils.gc_heap_size()\nheap_free = utils.heap_free()\nprint(\"Heap size: %d bytes, free: %d bytes\" % (mem_heap,heap_free))\nif mem_heap != 393216:\n    print(\"Decreasing GC heap size...\")\n    utils.gc_heap_size(393216)\n    reset()<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">3.4 Init LCD<\/h4>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python line-numbers\">lcd.init(freq=15000000)\nlcd.rotation(1)\nlcd.clear(0,0,0)\nlcd.draw_string(20,20, \"------Big Dolphin-----\", lcd.WHITE, lcd.BLACK)\nlcd.draw_string(20,40, \"----Digit Regconizing----\", lcd.WHITE, lcd.BLACK)\nlcd.draw_string(20,60, \"- CPU Frq: \" + str(cpu_frq) + \" MHz\", lcd.WHITE, lcd.BLACK)\nlcd.draw_string(20,80, \"- KPU Frq: \" + str(kpu_frq) + \" MHz\", lcd.WHITE, lcd.BLACK)<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">3.5 Load Mnist model<\/h4>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python line-numbers\">lcd.draw_string(20,100, \"- Loading model...\", lcd.WHITE, lcd.BLACK)\ntask_mnist = kpu.load(\"mnist.kmodel\")<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">3.6 Init Camera<\/h4>\n\n\n\n<p>This pre-trained Mnist model was trained with image size of 224 x 224, therefore, camera resolution must be configured at 224 x 224.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python line-numbers\">lcd.draw_string(20,120, \"- Loading Camera...\", lcd.WHITE, lcd.BLACK)\nsensor.reset()\nsensor.set_pixformat(sensor.RGB565)\nsensor.set_framesize(sensor.QVGA)\n# set to 224x224 input\nsensor.set_windowing((224, 224))\n# No vertial flip\nsensor.set_vflip(0)\n# No horizontal mirror\nsensor.set_hmirror(0)\nsensor.run(1)\nsensor.skip_frames(30)<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">3.7 Clear LCD<\/h4>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python line-numbers\">time.sleep(1)\nlcd.clear()<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">3.8 Main loop<\/h4>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python line-numbers\">while True:\n     # read camera\n     img_mnist = sensor.snapshot()\n     # Rotate image\n     img_mnist.set(hmirror=True, vflip=False, transpose=True)\n     # Convert to gray\n     img_mnist1 = img_mnist.to_grayscale(1)\n     # Resize to mnist input 28x28\n     img_mnist1 = img_mnist1.resize(28,28)\n     # Histogram equalization\n     img_mnist1 = img_mnist1.histeq(adaptive=True)\n     # Get histogram\n     img_hist = img_mnist1.get_histogram()\n     # Get optimal threshold\n     otsu_th = img_hist.get_threshold()\n     otsu_l = otsu_th.l_value()\n     otsu_a = otsu_th.a_value()\n     otsu_b = otsu_th.b_value()\n     # Thresholding image\n     img_mnist1 = img_mnist1.binary([(0, otsu_l), (-128, otsu_a),(-128, otsu_b)])\n     # Preprocessing pictures, eliminate dark corner\n     img_mnist1.strech_char(1)\n     # Generate data for ai\n     img_mnist1.pix_to_ai()\n     # Run neural network model\n     fmap_mnist = kpu.forward(task_mnist,img_mnist1)\n     # Get result (10 digit's probability)\n     plist_mnist = fmap_mnist[:]\n     # Get max probability\n     pmax_mnist = max(plist_mnist)\n     # Get the digit\n     max_index_mnist = plist_mnist.index(pmax_mnist)\n     # Print results\n     print(\"Detected number: %d \" % (max_index_mnist) + \", confidence: %.1f%% \" % (pmax_mnist<em>100))     # Show large picture to LCD     lcd.display(img_mnist,oft=(0,80))     # Show small picture to LCD     lcd.display(img_mnist1,oft=(0,50))     # Draw results to LCD     if pmax_mnist &gt; 0.55:         lcd.draw_string(8,8 ,\"Detected number: %d \" % (max_index_mnist),lcd.WHITE,lcd.BLACK)         lcd.draw_string(8,24,\"---&gt; Confidence: %.1f%% \" % (pmax_mnist<\/em>100),lcd.WHITE,lcd.BLACK)\n     else:\n         lcd.draw_string(8,8 ,\"Detected number: NaN \",lcd.WHITE,lcd.BLACK)\n         lcd.draw_string(8,24,\"---&gt; Confidence: %.1f%% \" % (pmax_mnist*100),lcd.WHITE,lcd.BLACK)\n     gc.collect()<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">3.9 Deinit KPU<\/h4>\n\n\n\n<pre class=\"wp-block-code\"><code lang=\"python\" class=\"language-python line-numbers\">kpu.deinit(task)<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">4. Test<\/h3>\n\n\n\n<p>Digit should be black color on white background for this model. Save the script above to board and run test.<\/p>\n\n\n\n<figure class=\"wp-block-video\"><video controls src=\"https:\/\/bigdolphin.com.vn\/wp-content\/uploads\/2021\/07\/mai.mp4\"><\/video><\/figure>\n\n\n\n<p>Full code and model can be downloaded from <a href=\"https:\/\/github.com\/bigdolphin\/maixduino\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/github.com\/bigdolphin\/maixduino<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Digi recognition with Maixduino using MNIST model and Micropython.<\/p>\n","protected":false},"author":2,"featured_media":1038,"comment_status":"open","ping_status":"open","sticky":false,"template":"single-with-sidebar","format":"standard","meta":{"gtb_hide_title":false,"gtb_wrap_title":false,"gtb_class_title":"","gtb_remove_headerfooter":false,"footnotes":""},"categories":[10],"tags":[31,30,32,37],"_links":{"self":[{"href":"https:\/\/bigdolphin.com.vn\/index.php?rest_route=\/wp\/v2\/posts\/1030"}],"collection":[{"href":"https:\/\/bigdolphin.com.vn\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/bigdolphin.com.vn\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/bigdolphin.com.vn\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/bigdolphin.com.vn\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1030"}],"version-history":[{"count":13,"href":"https:\/\/bigdolphin.com.vn\/index.php?rest_route=\/wp\/v2\/posts\/1030\/revisions"}],"predecessor-version":[{"id":1156,"href":"https:\/\/bigdolphin.com.vn\/index.php?rest_route=\/wp\/v2\/posts\/1030\/revisions\/1156"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/bigdolphin.com.vn\/index.php?rest_route=\/wp\/v2\/media\/1038"}],"wp:attachment":[{"href":"https:\/\/bigdolphin.com.vn\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1030"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/bigdolphin.com.vn\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1030"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/bigdolphin.com.vn\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1030"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}