Tag: digital

Posted on

Let’s build Metaverse with AI: What we have?

In the previous post about building metaverse with AI (link), I discussed the generic points of view, what we need and all the stuff like that. In this post, I am going to discuss about AI models we have which can be helpful in order to build the metaverse using AI and also possible pipelines.

Also, remember that in this particular post, I only will be discussing the AI models which I think can be helpful in building a virtual universe. So if your favorite AI isn’t in the list, accept my apologizes.

AI models for Metaverse

First of all, I think for building a virtual universe or metaverse using AI, we need these models:

  • Image generation models: These models will help us build everything imaginable. These are essential in pretty much every AI art project and of course, very useful in order to make the concept of our supposed Metaverse.
  • Music/SFX generation models: Imagine walking in a jungle. The landscape is pictured in your mind right? Now go a little deeper. You hear the sounds in your head, too. This is what we call soundscape in Ambient or minimalistic music (as I wrote about it before). Now let’s consider a metaverse we’re building, right? This newly made universe needs sounds. Without sounds, metaverse doesn’t mean anything. We need AI models in order to generate music, sounds and soundscapes for us.
  • Vision Language Models: These are important as well. In building the metaverse, we need everything to be as automated as possible. Basically, we need the matrix but in a good way. So a vision model can easily analyze a scene and generate respective prompts for sound generators.
  • 3D Generation models: And the question is why not? We try to make a complete 3D universe and we need to make 3D objects which let people make their desired universe, right? With AI, this will be a reality.

Now, let’s dive a little more in depth and look at what models we have access to!

Image Generators

If you ask me, this is the easiest type of model to find for this particular project. We have tons of proprietary options such as Dall-E 3 or Midjourney or even FLUX Pro. Which are all considered the best in the business.

In the open source side, we’ve got Mann-E, Stable Diffusion and other useful models as well, right? This means with a small search on the web, we can find out the best way of visualizing our dreams of a made-up universe.

Also, due to my research about different models and hosting services, hosting models on replicate or modal is very easy. For other types of hosting we may explore possibilities on CivitAI or Runware as well.

Music and Sound Effects generators

This is also not a rare thing. Although I am not really familiar with the music generation space and I only know Stable Audio LM and Meta’s Music Gen in open space, and Suno AI in proprietary space, I guess we already have the best in the business.

Vision Models

Well, I personally use Open Router to find out about the possibilities of these models, and being honest, the best model I could find for vision task was nothing but GPT-4o.

Although there are good vision models out there, but most of them are very generic or very specific and GPT-4o is right at the middle. We can use this model in order to describe different scenes in our metaverse. Also, we may utilize this model in order to be a guide through the metaverse or just help us build 3D objects or soundscapes.

3D Generation Models

Well these models are currently the rarest models in the list. We may need two approaches for this specific task:

  • Text to 3D: very similar to text to image, you just describe your scene or object, and get the 3D object. Although it may be a little buggy, but it will be a fun experiment to implement a model or pipeline for text to 3D. It will help the residents of our metaverse to generate assets of their choice as easy as typing what they have in their minds.
  • Image to 3D: This is also a possibility. Currently, I use TripoSR a lot for making different 3D objects, but I still couldn’t find the best input images or the best settings or hyper-parameter tuning for getting the best results.

With 3D generators, our workflow will become much much easier than what you may think. So we need another step, right?

What’s next?

Well, in the previous post we discussed the whole idea of metaverse and what we need to build one. In this one, we just discovered the AI tools we may be able to utilize. The next will be a study on how we can make a metaverse AI model at all.

It will be the most challenging part of the project, but in my honest and unfiltered opinion, it is also the best part!

Posted on

Let’s build Metaverse with AI : Introduction

It was 2021, the whole products under the flag of Facebook, went down for a few hours. I remember that most of my friends just started messaging me on Telegram instead of WhatsApp and also no new post or story was uploaded on Instagram.

A few hours passed, everything went back to normal, except one. Zuckerberg made a huge announcement and then told the whole world Facebook will be known as Meta and he also announced the Metaverse as a weird alternate life game where you can pay actual money and get a whole lot of nothing.

I personally liked the idea of metaverse (and at the same time, I was a co-founder of ARMo, an augmented reality startup company) so you may guess, it was basically mu job to follow the trends and news about metaverse and what happens around it.

It’s been a few days I am thinking about metaverse again. Because I have a strong belief about the whole thing becoming a hype again. Specially with this bull run on bitcoin and other currencies. I also concluded that metaverse has a big missing circle, which I’m going to discuss in this post.

A little backstory

Since I started Mann-E, as an AI image generation platform, a lot of people messaged me about connecting the whole thing to the blockchain. Recently, I just moved the whole payment system to cryptocurrencies and I’m happy of what I’ve done, not gonna lie.

But for being on the chain, I had different thoughts in mind and one of them was an ICO, or even an NFT collection. They may seem cool but they also always have the righteous amount of criticism and skepticism as well. I don’t want to be identified as a bad guy in my community of course, so I left those ideas for good.

As you read prior to this paragraph, I have a history in XR (extended reality) business and currently, I have my own AI company. I was thinking about the connection of Metaverse and AI, and opportunities of both!

Before going deep, I have to ask a question…

What did we need to access the metaverse?

In 2021, when it was the hot topic of every tech forum, if you asked Okay then, how can I enter the metaverse? No one could answer correctly. At least in Iranian scene, it was like this.

I did a lot of research and I found these to enter a metaverse of choice:

  • A crypto wallet: Which is not a big deal. Pretty much everyone who’s familiar with tech and these new trends, owns a crypto wallet. They’re everywhere. You can have them as web apps, native apps, browser extensions and even in hardware form. If you want to waste a few hours of your life, you also can build one from scratch.
  • Internet browser: Are you kidding me? We all have it. Currently most of the applications we’ve used to install on our computers turned into SaaS platforms. We need to have a good browser.
  • A bit of crypto: The problem in my opinion starts here. . Most of these projects however had a token built on ETH network (or accepted Ethereum directly) but some of them had their native currencies which were impossible to buy from well-known exchanges and as you guessed, it increased the chance of scam! But in general it was a little odd to be forced to pay to enter the verse without knowing what is happening there. I put an example here for you. Imagine you are in Dubai, and you see a luxurious shopping center. Then you have to pay $100 in order to enter the center and you just do window-shopping and leave the shopping center disappointed. It’s just a loss, isn’t it?

But this is not all of it. A person like me who considers him/herself as a builder needs to explore the builder opportunities as well, right? Now I have a better question and that is…

What we need to build on Metaverse?

In addition to a wallet, a browser and initial funds for entering the metaverse, you also need something else. You need Metaverse Development Skills which are not easy to achieve.

If we talk about programming side of things, most of the stuff can be easily done by using libraries such as ThreeJS or similar ones. If you have development background and access to resources such as ChatGPT, the whole process will not take more than a week to master the new library.

But there was something else which occupied my mind and it was 3D Design Skills which are not easily achievable to anyone and you may spend years to master it.

And this is why I think Metaverse needs AI. And I will explain in the next section.

The role of AI in metaverse

This is my favorite topic. I am utilizing AI since 2021 in different ways. For example, I explained about how I could analyze electrical circuits using AI. Also if you dig deeper in my blog, you may found I even explained my love of YOLOv5 models.

But my first serious Generative AI project was the time GitHub’s copilot becoming a paid product and I was too cheap to pay for it, so I build my own. In that particular project, I have utilized a large language model called BLOOM in order to generate code for me. It was the beginning of my journey in generative artificial intelligence.

A few months after that, I discovered AI image generators. It lead me to the point I could start my own startup with just a simple ten dollars fund. Now, I have bigger steps in mind.

Generative AI for metaverse

There is a good question and that is How can generative artificial intelligence be useful in the metaverse? And I have a list of opportunities here:

  • Tradebots: Since most of metaverse projects offer their own coin or token, we may be able to utilize AI to make some sort of advice or prediction for us. Honestly, this is my least favorite function of AI in the metaverse. I never was a big fan of fintech and similar stuff.
  • Agents: Of course when we’re entering the matrix, sorry, I meant metaverse, we need agents helping us find a good life there. But jokes aside, Agents can help us in different ways such as building, finding resources or how to interact with the surrounding universe as well.
  • Generating the metaverse: And honestly, this is my most favorite topic of all time. We may be able to utilize different models to generate different assets for us just in order to build our metaverse. For this particular one, we need different models. Not only LLMs, but image generators, sound generators, etc.

What’s next?

The next step is doing a study on every resource or model which can be somehow functional or useful in the space. Also we may need to explore possibilities of different blockchains and metaverses in general. But first, the focus must be on AI models. The rest will be made automatically 😁

Posted on

FrontBricks, my LLM-based weekend project which is inspired by Vercel’s V0

Since 2022, there is a hype of generative artificial intelligence and it resulted in a bunch of cool projects. Although a lot of us may remember that Github’s copilot was much older. Those days, I wrote an article about how I was too cheap to pay $10 a month for copilot, so I made my own!

That was somehow the beginning of my interest in AI field. I spent around four years in this field and like most of us, I tried to utilize different tools and products. In this article, I’m talking about FrontBricks which is my newest product and how it started as a weekend project!

A little bit of history

In 2023, I launched Mann-E which is an AI image generator based on its own models (and more information is provided in the website). A few months ago, I also launched Maral, which is a 7 billion parameter LLM specialized for the Persian language (the language I speak).

Also, around a month ago, I did some tests with brand new LLMs such as LLaMa 3, in order to make Mann-E Search which can be somehow an alternative to Perplexity but with a little difference (it doesn’t provide a chat interface).

I guess this can clarify how I am drowned in AI space and how much I love generative AI! Now we can talk about FrontBricks!

What is FrontBricks?

You may be familiar with Vercel’s V0 which is a generative AI tool helping people generate frontend components. I liked their idea, and I joined their waitlist and a couple days later, I got access to the platform.

It was a cool experience, and some sparks formed in my head. I found out that pretty much all LLMs are really good at the task of code generation, and we can utilize one to generate the code and use another one in order to find out if the code is valid or not.

This was my whole idea so I sat at my desk and started to code a basic tool to send my prompts to OpenAI’s API in order to generate and then another one to do the validation using LLaMa 3 70B and GPT-4 as well (I used OpenAI again).

I also found another bottleneck, which was JSX code generation. I did a little bit of research and I found that is not really a big deal and using the power of Regex and text manipulation, it’s easily possible to turn pure HTML to JSX!

I wrote pretty much everything, so I just switched to my work environment, created a simple rails app and then connected it to my backend module. Now, I have a platform which can be an alternative to Vercel’s V0!

Today, I am just announcing frontbricks, but I have to say before this post around 211 people gave me their email addresses to put them in the list of early adopters and I gave them access to the platform earlier this week!

My birthday (May 30th) was in this week, so I guess it can also be a bit of surprise for my friends and the community.

How can I access FrontBricks?

Well, it is easy. You just need to go to frontbricks.com and create an account (sign up link). Then you just need to confirm your email and boom, you have unlimited access to FrontBricks, completely free of charge!

You can generate a component, then improve it and every time you felt you need a new component, you easily can choose to create a new code snippet. It is as easy as drinking a cup of tea.

Future Plans

Since this project isn’t monetized yet, the very first thing coming to my mind is a way to monetize it (you still can donate in crypto through this link). A good business model can help this project be much better.

I also am thinking of releasing an open source model based on the data provided on FrontBricks, because one of the reasons I coded this project is just that I couldn’t find a model specialized for front-end generation!

These are my concerns for now. If you have any other ideas, I’m open to here.

Conclusion

I have a haystack of ideas in my mind, and if I find enough time, I implement them. Mann-E and FrontBricks are just two of projects I just made and to be honest, Mann-E with around 6000 users and more than 50,000 generated images, is somehow one my most successful projects.

FrontBricks has potential, but I guess I can’t keep it up alone. I’m open to technical and business ideas as well. So if you have any ideas in mind, feel free to send me a message, my email is haghiri75@gmail.com 😁

Posted on

Why I love YOLOv5?

I am a big fan of Nicholas Renotte’s channel on YouTube. I also love computer vision and its combination with deep learning. A few months ago, Nicholas posted this video, which is about YOLOv5. I usually am too lazy to watch videos which are longer than 15 minutes and I watch them in a few episodes. But this video made me sit behind the laptop screen for over an hour and I’m sure I won’t regret it.

So let’s start the article and see where this story begins. As I mentioned earlier, I love computer vision specially when it’s combined with deep learning. I believe it can help us solve very complex problems of our projects with ease. My journey in world of these YOLO models have started almost a year ago, when I wanted to develop a simple object detection for detecting street signs.

Firstly, I found a lot of tutorials on darknet based training but l did not manage to get it to the work, specially since I have a mac, it could be a very realistic nightmare. So I guess YOLOv5 was a miracle. In this article, I am going to explain why I love YOLOv5 and why I prefer it to other YOLO versions.

What is YOLOv5?

According to their github repository, YOLOv5 is a family of deep learning models which is essentially trained on Microsoft’s COCO dataset. This makes it a very very general-purpose object detection tool which is fine for basic research and fun projects.

But I also needed to have my own models because I wanted to develop some domain-specific object detection software. So I realized they also provide a python script which helps you fine-tune and train your own version of YOLOv5.

So I basically fell in love with this new thing I have discovered. In the next sections, I will explain why I love YOLOv5!

Why I love YOLOv5?

Firstly, I invite you to see this chart, which shows the comparison of YOLOv5 with other commonly used object detection models:

And since there’s been a controversy about YOLOv5 claims about training time, inference time, model storage size, etc. I highly recommend you read this article on Roboflow’s blog.

So we can conclude the very first thing which made me happy is the speed and that’s right. The second thing by the way is the fact I am lazy. Yes, I am lazy and I know it.

I always tried to compile darknet and use it for having a YOLOv4 model and make my projects on top of YOLOv4 but when I saw how hard it can get and since I have a mac and I didn’t really want to fire-up an old computer for these projects, I was looking for something which does everything with a bunch of python scripts.

Since I discovered the YOLOv5, I started working with it and the very first project I have done was this pedestrian detection for a self-driving car.

Then, I started doing a lot of research and asking about what I can do with YOLOv5. I find out I can do pretty much anything I want with ease as they provided a lot of stuff themselves. Isn’t that good enough? Fine. Let me show you another youtube video of mine which I solved my crop problem with their internal functions.

If you’re not convinced yet, I have to tell you there is a great method which is called pandas in this family of models.

As the name tells us, it really outputs a pandas dataframe which you can easily use data from that dataframe. Let me set a better example for you. Considering we want to find out which plants are afflicted and which ones are not in a drone footage.

By using this method, we can simply make an algorithm which counts the amount of afflicted ones in a single frame, so we can easily find out how many afflicted plants we have in a certain area. The whole point here is that we have statistically right data for most of our researches.

The other example would be the same as my pedestrian detection system. We can command the car to get data first from the cameras to make sure we’re dealing with pedestrians and second get data from distance measurement system (which can be an Ultrasonic or LiDAR) to make sure when it should send braking command.

Conclusion

Let’s make a conclusion on the whole article. I love YOLOv5 because it made life easier for me, as a computer vision enthusiast. It provided the tools I wanted and honestly, I am really thankful to Ultralytics for this great opportunity they have provided for us.

In general I always prefer easy-to-use tools and YOLOv5 was this for me. I need to focus on the goal I have instead of making a whole object detection algorithm or model from scratch.

I finally can conclude that having a fast, easy-to-use and all-python tool for object detection was what I was always seeking and YOLOv5 was my answer.

I am glad to have you as a reader on my blog and I have to say thank you for the time you’ve spent on my blog reading this article. Stay safe!

Posted on

How to make video games like movies!

It was a long time that I did not write any thing in this blog. Now, I decided to write a topic about “video games” (as I wrote in my Persian blog). I was member of a game development team for about three months and I learned a lot. At least, I know the way they were doing the job was “How to not make a video game”. So, When I left the team, I decided to research about game development process. In this topic, I explain everything I found (experience and research result!)

When I was in the team…

It was in October (2017), a person sent me a message in Telegram, and the message was like this :

Mr. Haghiri, we need a musical composer for our game, we did a search and we found you. Please come here Wednesday 4:00 PM to talk about the project and your role.

Wednesday, I went to their office. That guy greeted me so nicely and started talking to me, about their project. I found the game is a horror game (horror games are popular in Iran, but there’s no “good” horror game “made in Iran”.) and it made me happy! Because it was the first time I heard about an indie team decided to make such a great game. They let me two weeks to research about “Sound and Music” in Unity Engine and I did it. I composed two pieces and also I tried to learn some tools for mixing and mastering sounds in the game (But it wasn’t actually my role, I just did that as a sample!).

After two and half months, they said “Mr. Haghiri, you don’t do what we wanted you to do”, anyway, they haven’t paid me even a “rial” in those days and expected me do great music composition, and they also wanted me to do what wasn’t my actual role. And this was not a good experience actually. But in those two months, I learned Unity game engine and I also met other “game developers”.  And, I decided to publish my experience on my blog.

Why movies?

Recently, I read a book called Making Short Films : Complete Guide from Script to Screen, by Clifford Thurlow. In the book, I found great names like “Salvador Dali” or “Charlie Chaplin”, and also great movies and books also mentioned in the book, like “One Flew Over Cuckoo’s Nest”. Everything was perfect, In I think about process of making a video game! It’s so similar to process of making a movie!

But I realize game and movie have lots of differences. The biggest difference is that games are interactive and players interact with the environment or other characters, but movies are not. Anyway, the main procedure – I mean “writing” – is %90 the same! So, I decided to mention some movies I watched, then tell you how I make a video game like them!

Great movies gave me ideas!

An Andalusian Dog

Movie is written by Louis Bunuel and Salvador Dali. I think these names are enough. But, after I watched the movie (it’s now available on YouTube and other video-sharing platforms, you can easily find and watch it), I discovered “product of a melancholic and depressed mind”. And both “Melancholia” and “Depression” are good subjects for a story or game.

Phantom of The Opera (Musical)

The book is about a musician, in a better word, a “genius” and smart person who hided himself. Andrew Lloyd Webber, just made that sad and creepy story to a romantic story by his music! First time I watched the movie, I could not understand it well. Because it’s not following the book’s story, and I’m not a native English speaker! I watched that movie 4 times. I watched the live performance 2 or 3 times and finally, I got the concept.

It has “Misanthropy” and “Romance” at once. I think these two things are also good for people who want to make video games (Please go and check “INSIDE” by Playdead! It’s the Misanthropy! Pure Misanthropy!)

School of Rock

It’s a bit different, School of Rock is comedy, and it’s also attractive for children. Because the topic is about a guy who teaches a bunch of 4th grade children to ROCK! Yes, he teaches them how to play electric guitars, bass, drums and keys. I think the concept of “Music” can be a good idea, too! Specially if you plan for a game like “Guitar Hero”.

Only Lovers Left Alive

If you like Vampires, please watch this movie. This movie has no “teenager” content, but it still tells stories about two vampires who married for centuries. Movie is directed by Jim Jarmusch and he also made music for his movie, in his rock trio SQRL. This movie has two things for game developers. It’s and independent movie and it can be a good idea bank for indie developers and also, it has “Romance” and “Fear” and “History”. All of these three factors can make a game great!

Ok, I talked a lot about movies! Let’s find “how can I make a game like a movie?”

Finally, let’s make a game like a movie!

The most basic thing you must have for a movie is “plot”. Plot is the main idea, it’s developed and tells people your concept, but it’s not a completed “script”. Writing plot in both games and movies is the same. To write a good plot, you need to study and read books, scripts and other plots. I prefer printed (and Persian) books in this case.

You will need research on the topic you want to write a plot about. For example, if you want to write about a middle eastern civilization (for example : Sassanid Empire or Ottoman Empire), you have to read history of Iran, Turkey, Afghanistan, Iraq, etc. If you want to write a plot about Satan, you have to search about Satan’s role in Judaism, Christianity and Islam, and find which belief is closer to what you want. So, you need to have a background.

After you wrote the plot, you have to write the script. But, writing script is different here, you have to clarify where and when the scene is interactive and when and where is not. The best example, is “Bioshock Infinite”. You know why? Because when you’re not interacting with the environment, you still can move camera and see what happens around you. I’m not a good script writer (But I try to be!) and I will write a post about how to write script from plot, when I manage to do that.

After you wrote the script, you have to make your basic ideas in the engine. Please! Please! Please! Call an experienced game developer before doing that, because the experienced one can help you find experienced character/concept and environment artist and game-play designer. After that, I can say you’re ready to start making your game! With a good team, you can make a good game!

Finally, I wrote this article but it wouldn’t be the last article about games in my English blog. I try to continue this, because I even couldn’t find good articles about being “game script writer” or “game director” even in English! I hope you like my post 🙂

 

 

Posted on

How to be a hardware engineer

A computer, is made up of hardware and software. Lots of people like to write and develop software, so the internet is full of topics like “How to be a software engineer” or “how to write a computer program” and also, there is a lot of great and free tutorials on programming languages and frameworks. But, hardware engineering guides on the internet, are not as many as software’s.

In this post, I really want to write about “How to” be “a hardware engineer”. As it’s my favorite field in computer science and I study it in the university. So, if you want to be a computer hardware engineer, just read this and start searching about your favorites.

Definition

A computer engineer in general, is an engineer who designs, develops and tests computers. Both hardware and software. But, there are different types of “Computer engineer”. For example, I am a hardware engineer, my friend is software engineer, my other friend is network engineer, etc.

So, a hardware engineer is a computer engineer, who knows electronic and electrical engineering (I know, some of electronics/electrical engineering courses are taught to students of software engineering). But, if we go further, we’ll find that the “Hardware Engineer” is who “combines computer science and electronics”. So, this is what he/she does, learns computer science and electronics, then combines them and makes computer hardware.

Expertise and Fields

There are these fields in hardware engineering, and people can learn one of them to find a job or start their own business.

  1. Computer architecture :
    a computer architect, is a computer scientist who knows how to design logical level of a computer. In my opinion it’s the most-needed role in a team who design a new computer. When you study computer architecture, you learn mathematics behind the computer hardware, and also you learn how to design different logical circuits, and connect them together and make a computer at the logical level.
    So, if you love mathematics, and love computer programming and in general, computer science, this is your field. to learn more, you also can read my book (link)
  2. Digital Electronics :
    This one, is actually my favorite. Digital electronics is about implementation of logical designs. So, with digital electronics, you can make what you have designed using logic and mathematics.
    But for this field of hardware engineering, you need to know electronics and electrical engineering. So, if you like digital electronics, start studies on electrical engineering today!
  3. Digital Design :
    This part, is also my favorite, and I really like it more than digital electronics. It’s about simulation and synthesis, so you can really ‘feel’ the hardware you have designed.
    But how is it possible? The “Digital Designer” is a person who’s familiar with simulation and synthesis tools. Such as hSpice, LTSpice, ModelSim and languages such as Verilog or VHDL. It’s about design and programming, so if you’re a software engineer who wants to learn hardware engineering, this field is yours! But, usually hardware engineers learn this field after learning digital electronics.
    If you like to be a digital designer, start improving your programming and algorithm design today, it’s important to know how to code, and how to write an algorithm.
  4. Microcontrollers and Programmable devices :
    This field is not actually “Hardware Engineering”. But, it’s still dealing with hardware. Actually, it’s not “design and implementation” of the hardware, it’s “making hardware usable”. When you program an ATMega32 chip for example, you make a piece of silicon usable for driving motors and sensors.
  5. Artificial Intelligence :
    And finally, the common field! Artificial intelligence is the widest field in computer science. It’s applicable and functional in all fields of computer science.
    But how we can use it in hardware engineering? We can use AI to improve our hardware, for example our architectures can be analyzed by AI. We can make a hardware which has AI to solve problems (a.g. ZISC processors). And finally, we can use AI in Signal processing, which is usually referred as a field of hardware/electronics/audio engineering.

Start point

I tried to explain start points in previous part, but now, I explain general start point. To learn hardware engineering, you need to know discrete mathematics and logic. Also, you need to know how to show boolean functions like electrical circuits (a.k.a logical circuits). After learning these, you need to get familiar with computer architecture, and at the end, you can choose your favorite field. I have experiences in all fields (except AI and Signal processing) but my most favorites are digital design and digital electronics.

It all depends on you, to choose which field, but you need those basics and fundamentals to understand it. Also, if you improve your knowledge about computer science in general, you will be a successful computer engineer!

Good Luck 🙂

 

Posted on

Importance of IoT

For the last 25 years, a ghost is flying over our world, it’s the ghost of “Internet”. From 80’s, internet became one of the most used tools for international communications, such as knowledge sharing. So, a lot of people started using internet to share knowledge, to do research or even to have fun. We can use internet in many different ways and this is why internet is important. You know, humans can’t live without air and water, but actually, electricity and internet are as important as them!

In 1982, a truck connected to the internet, it was the first attempt to connect non-digital things to the internet. Years after that, someone asked people of the world to call “connecting non-digital things to the internet”, Internet of Things or IoT for short. Actually a computer or a mobile phone is designed to be connected to internet, so we can’t call that IoT. But a wheelchair, a cigarette lighter, a coffee maker or even an oven are not designed to be connected to internet, so if we connect them to the internet, it’s IoT!

Concepts and Applications

The concept of IoT is to make everything around us online. You can control your coffee maker from work, and when you arrive home, you can have a cup of warm coffee. But, This is not the only concepts. It comes to make our lives safe, it comes to make our things accessible from everywhere. And about applications? As I told, the online-coffee maker is one of the best examples of IoT applications.

The main application of IoT is controlling everything remotely. Before IoT, remote controls used infrared to control lamps, outlets, TV’s, etc. But today, thanks to IoT, we can do that from longer distances, using the internet. For example, your home is in Iran, and you are in a vacation in Armenia, but you can control everything from Armenia! Everything you need is a reliable internet connection.

The importance

Now, let’s talk about the importance. Why IoT is important? is it anything except connecting devices like ovens and coffee makers to the internet? of course it is! It is actually the future of the hardware engineering and even the online world. Hardware engineers like me should learn IoT, because in near future, every single home accessory will become online, every streets of cities will be online, all farms will be online!

So, we need to learn IoT, but how? learning IoT is easy. You only need to know how to program microcontrollers, how to setup a development board (at all, hardware engineering), how to connect your board to the network and how to implement remote controls and connections. For example, you can use a NodeMCU board and a bunch of LED’s to make a bedside lamp, it’s even easier than writing a simple computer program for daily accounting.

Communication with other fields of computer science

IoT can’t be a complete knowledge/technology alone. All IoT experts have to know about one of fields of computer science. For example, a person who designed a system for traffic control, needs to store data from traffic lights or cameras, and then he/she needs to do data mining on that. Because this thing can be helpful in predictions about traffic. Or, if you design that simple bedside lamp, you have to write at least a good single-page interface for it. These are not basic things, and you can’t do all of them alone. To be honest, an IoT team is made up of some professionals/experts. What they do is cooperating about these issues.

Good Luck 🙂

Posted on

Reverse engineering of 8086, from a calculator to the most used processor

If you have a laptop or desktop computer, you probably use a 8086-based CPU, or one implementation of x86 family to be exact. For example, I have a Lenovo laptop with a Core i5 CPU, which is based on x86 architecture. In this article, I want to talk about x86 architecture, and to explain how it works, I just start with the simplest one : 8086.

8086, is probably the first general-purpose processor made by Intel. This is why it’s famous, and in a lot of cases, people prefer to use it or study it. Everything is well-documented and also there are billions of tutorials and examples on how to use it! For example, if you search for “Interfacing Circuits”, you will find a lot of 8086-based computers made by people, connected to interface devices such as monitors, keyboards or mice.

Before we start reverse engineering, and make our simple x86-compatible computer, let’s take a look on the machine code structure of 8086. In this case, we just review Register addressing mode, because this mode is easier to understand or re-implement.

In this case, we can only take a look on a two-byte (or 16 bit) instruction code. Our instruction code looks like this :

Byte 1 Byte 0
|Opcode|D|W| |MOD|REG|R/M|

What are these? and why we should learn this? As we want to reverse engineer 8086 architecture and learn how it works, we need to know how this processor can understand programs! So, let’s check what are those fields in these bytes :

  • Opcode : a 6-bit number, which determines about operation (for example ADD, SUB, MOV, etc. )
  • D : Determines source or destination operand. To make reverse engineering process simple, we consider that as constant 1. So, REG field in byte 0 is always destination.
  • W : Determines data size, and like D, to make reverse engineering process simple, we consider it as a constant 1. So, we only can do operations on 16 bit numbers.
  • MOD : Determines mode, as we decided before, we only model the register addressing mode, so we need to consider mode as constant 11.
  • REG and R/M : REG shows us source, R/M shows us destination. Please pay attention, we made this special case because we are going to model register addressing mode. For other modes, we can’t consider R/M as destination.

Now, we learned how 8086 can understand programs, for now, we have some instruction code like this :

Opcode D W MOD REG R/M
xxxxxx 1 1 11 xxx xxx

Let’s assign codes to our registers. As we decided to simplify our reverse engineering process, and also we decided to use only 16 bit registers, I prefer to model four main registers, AX, BX, CX and DX. This table shows us codes :

Code Register
000 AX
001 CX
010 DX
011 BX

As you can see, now we are able to convert instructions to machine code. To make reverse engineering process even simpler, we can ignore “MOV”, but I prefer to include MOV in my list. So, let’s find opcodes for MOV, ADD and SUB.

Opcode Operation
100010 MOV
000001 ADD
010101 SUB

Now, we can convert every 8086 assembly instruction using the format we have. For example this piece of code :

MOV AX, CX 
MOV BX, DX
ADD AX, BX

Now, if we want to convert this piece of code to machine code, we have to use the tables we made. Now, I can tell you these codes will be :

1000101111000001
=> 0x8bc1

1000101111011010
=> 0x8bda

0000011111000011
=> 0x07c3

Now, we can model our very simple x86-based computer. But a note to mention, this has a lot of bugs! For example, we can’t initialize registers, so we need to study and implement immediate addressing mode, also, we can’t read anything from memory (x86 is not a load/store architecture, and it lets people work with data stored in memory directly!). But I think, this can be helpful if you want to study this popular processor or do any projects with it.

Posted on

Microcontrollers, Design and Implementation released!

It was about two years I started serious study on computer architecture. In these years, I learned a lot and I could simulate and implement a microprocessor, similar to real ones. In Summer 2016, I decided to share my experience with others. Then, I started writing this book. This book has seventeen chapters, and after reading this book, you will have a concept of computer architecture.

Chapters

  • License – Licensing and Copyrights
  • Introduction – A quick review of the book, defining target audience of the book.
  • Chapter 1 : What’s a microcontroller? – This chapter, defines a microcontroller. After reading this chapter you’ll understand the internal parts of a microcontroller. It’s completely theory, but you need the concepts.
  • Chapter 2 : How to talk to computer? – In this chapter, we have a quick view on programming and then, machine language. We determine the word size of our processor in this chapter.
  • Chapter 3 : Arithmetic Operations – This chapter focuses on arithmetic operations in base 2.
  • Chapter 4 : Logical Operations – This is all about boolean algebra, the very basic introduction to logical circuits.
  • Chapter 5 : Logical Circuits – Our journey starts here, we learn how to make logics using NAND in this chapter, and then, we learn the logic gates.
  • Chapter 6 : Combinational Circuits – This chapter is where you learn how to combine simple logics together and make more complex logics. Actually, you learn how to implement Exclusive OR and Exclusive NOR using other gates.
  • Chapter 7 : The First Computer – In this chapter, we make a simple Addition Machine.
  • Chapter 8 : Memory – In this chapter, we just take a look on sequential circuits.
  • Chapter 9 : Register File – After we learned sequential circuits, we make registers and then, we make our register file.
  • Chapter 10 – Computer Architecture – In this chapter, we’ll learn theory and basics of computer architecture and organization .
  • Chapter 11 – Design, Advanced Addition Machine – In this chapter, we add memory blocks to our addition machine.
  • Chapter 12 – The Computer (Theory) – In this chapter, we decide about what our computer should do. Actually, we design a simple ISA.
  • Chapter 13 – Arithmetic and Logical Unit – Now, it’s time to design our ALU.
  • Chapter 14 – Program Structure – In this chapter we decide about programming and machine language, and we design a simple instruction code.
  • Chapter 15 – Microcontroller – And finally, we add the RAM to our ALU, and we’ll have our simple microcontroller.
  • Chapter 16 – Programming and Operating System – In this chapter, we actually talk about the software layer of computers.
  • Chapter 17 – The Dark Side of The Moon – The final chapter, is all about making real hardware, we take a look at transistors, integrated circuits and HDL’s here.

Link to PDF File : Download

Posted on

Hexadecimal Decoder (Seven Segment)

You may use a seven segment to show time, temperature, voltage, etc. But, when you want to use it as an interface, specially on a simple computer, usual decoders (e.g. 7447) are not the best choice. You may need a shift-register or you may need to make a specific decoder for your computer display.  If you watch Ben Eater’s video about EEPROM,  You will find an Electrical Erasable ROM can be your decoder. But how? Imagine we have a common-cathode seven segment display. Then, We will have something like this for number zero :

dp g f e d c b a
0 0 1 1 1 1 1 1

So,  if we want to show a hexadecimal number, what will we need? Of course, we need to find which pins of our common cathode should be on for a specific number. In case of common anode displays, pins should be off.  Now, let’s design our Truth Table.

Number dp g f e d c b a
0 0 0 1 1 1 1 1 1
1 0 0 0 0 0 1 1 0
2 0 1 0 1 1 0 1 1
3 0 1 0 0 1 1 1 1
4 0 1 1 0 0 1 1 0
5 0 1 1 0 1 1 0 1
6 0 1 1 1 1 1 0 1
7 0 0 0 0 0 1 1 1
8 0 1 1 1 1 1 1 1
9 0 1 1 0 1 1 1 1
A 0 1 1 1 0 1 1 1
b 0 1 1 1 1 1 0 0
C 0 0 1 1 1 0 0 1
d 0 1 0 1 1 1 1 1
E 0 1 1 1 1 0 0 1
F 0 1 1 1 0 0 0 1

This is our truth table. Fortunately, Logisim can make a circuit from a truth table, but I made this circuit before. It looks like this :

The inputs W, X, Y and Z represent the BCD input. But how can make this thing? if you are interested to make one of these for real, you can use a 2816 chip, which is actually an EEPROM.