Week 6: A Week Without My Code Breaking Is A Good Spring Break

“Not all artists are Chess players, but all Chess players are artists.” -Marcel Duchamp

Happy Spring Break!!! I hope y’all had fun this week!!! I actually enjoyed my spring break for once (as I have no AP exams to study for), but I still made a bit more progress on my senior project as well. Today will be a slightly shorter blog, so hopefully I don’t take up too much of your time. Anyways, let’s get started.

1. Studying MRTK Buttons
2. Reworking The Input System
3. Adjusting Dimensions
4. Plans For Next Week

MRTK Buttons

In my mentor’s old project file, I found lots of demo scenes that show how each button works. I initially thought they came with importing the Mixed Reality Tool Kit to Unity, but turns out, I was wrong. So, I had to go export them and then re-import them back into my new AR chess project. 

Now, I have a few samples of pre-made MRTK buttons from Microsoft, yay! The question that remains is, how do I customize them to fit what I need them to do? Sadly, these buttons turned out to be more complicated than I expected, so I had to do a bit of reading.

Understanding MRTK Buttons

Luckily for me, Microsoft has a lot of documentation regarding how they implemented these buttons [1, 2]. After reading through them, I decided to use the Collider based buttons, since they are shaped as squares and can easily be arranged so that they fit under the chess board. After adjusting the size (which was actually pretty difficult), here’s what it looks like: 

For some reasons, the blue plate on the button keeps disappearing. I think that’s just how it was implemented. The blue only shows up if the user is “focusing” on that particular button, but ideally, I would like to disable that feature, since it looks uneven when I have 64 buttons all together. Anyways, that will be a task for next time.

New Input System

As I mentioned last week, my most pressing goal is to replace the mouse input system with a hand gesture input system. After adjusting the size of the button so that they fit under each square, I replaced the cubes I had from last week with the MRTK buttons. Here is what it looks like: 

Again, the buttons are there, it’s just a bit hard to see because the blue back plate keeps disappearing.

Implementing The Buttons

First of all, because it’s quite unwieldy to duplicate the buttons 64 times, I wrote a script that creates the buttons via code. Then, the TileGridObjectCollection script that I talked about last time will take care of arranging them. Every button has an index assigned to it, from 0 to 63. This index is really useful because I can access its corresponding row and column by dividing by 8 and then taking the remainder. I also made another script, which essentially passes the id (and also the coordinates) to the BoardManager whenever the user clicks on the button. 

After that, I’m all set! Once the BoardManager receives the coordinates of the button pressed, it can just do whatever it did before. I did had to make one slight change though. Previously, the coordinate of the selected square was based on Cartesian coordinates, so I manually converted it to grid coordinates. However, now, the coordinates that the BoardManager receives from the buttons are already in grid coordinates, so that conversion is not necessary.

It’s a bit difficult to show that everything is working through screenshots, but trust me, it all works.

One Slight Issue

There is a tiny problem with using the hand gestures as input. As you can see in the diagram above, many of the buttons are actually covered up by chess pieces. Thus, to be able to select that square, the user would need to click on the margins that are not covered up. This makes it slightly more inconvenient when moving the pieces. However, I don’t think this is such a major issue, but it is definitely something I will be working on eventually.

Adjusting Dimensions

Now, for my training system to be “realistic,” the size of the chess board itself must be accurate. Previously, I was working on a chess board that was 8 meters by 8 meters — obviously too big. So, the final thing I did this week was make my chess board adjustable in size. It wasn’t too difficult, as all I had to do was multiply the Vectors used to create the chess board by the tileSize variable. 

However, I also wanted to be able to automatically adjust the size of the chess pieces themselves the same way. In my code, after instantiating each chess piece, I would then set the scale to the appropriate Vector3. Now, by adjusting the tileSize variable, the size of the chess pieces adjust as well. Perfect! 

There is one issue though. Adjusting the size of the buttons isn’t as simple as changing the scale. In fact, not only do I need to change the scale in several different places, I also need to click a button that activates a complicated script to make sure the colliders are in the right positions. So, even though I’m able to adjust the dimensions of the board and pieces pretty easily, I would still need to manually adjust the size of the buttons, which is not ideal.

Plans For Next Week

Next week, I will finally start implementing the database portion of my project. I want to be able to store annotated games within a database for players to use to study, so getting this done is pretty important. Luckily, I already have some code for that from my Capstone CS class, but I still need to translate it into C# and also write additional code to process chess notational moves. If I have time, I will then work on making the buttons more accessible. 

Anyways, that’s all for today! There are only a few more weeks before the final presentation, but I think I’m making decent progress. Hopefully very soon, I’ll actually be able to test this system out on a headset. Until then, go enjoy your break!!!

References

1. cre8ivepark. “Buttons – MRTK 2.” Buttons – MRTK 2 | Microsoft Learn, Microsoft, 23 June 2022, Https://Learn.Microsoft.Com/En-Us/Windows/Mixed-Reality/Mrtk-Unity/Mrtk2/Features/Ux-Building-Blocks/Button?View=Mrtkunity-2022-05.
2. CDiaz-MS. “Interactable – MRTK 2.” Interactable – MRTK 2 | Microsoft Learn, Microsoft, 1 Aug. 2022, Https://Learn.Microsoft.Com/En-Us/Windows/Mixed-Reality/Mrtk-Unity/Mrtk2/Features/Ux-Building-Blocks/Interactable?View=Mrtkunity-2022-05.