Week 3: Mickey Docks at Shore

March 15, 2024

Mickey Ears

Good news, Mickey has finally “docked”, and you can see the long-awaited point cloud!

This week, I have found a way to run OpenSFM commands by using a Docker image kindly provided by Dr. Jeff Liu, technical staff member in the Humanitarian Assistance and Disaster Relief group at MIT Lincoln Lab and my instructor from the Remote Sensing for Disaster Response course at MIT Beaverworks. Docker containers are isolated containers of images dictating the instructions for all the code and libraries for running an application. Since Docker containers are independent of your local system, they can easily be shared with others to solve the problem of “it doesn’t work on my computer.” Here’s how I ran a Docker container, which contains an image with the instructions for compiling an environment with OpenSFM already built. I had to bind my local server to the exposed Docker port (-p 8888:8888) in order to be able to open the Jupyter Notebook link generated, which Professor Guzun gave me the command for.
opensfm docker command

Once I opened the link, the Jupyter Notebook initially has an empty folder “work.” I added a Python notebook inside and another folder called “mickey_with_ears” with a folder called “images” where I uploaded all of my images.

I put the Mickey ears on top of another Mickey because I thought the images had a problem at first. When I uploaded the images, I didn’t make a folder called “images” inside the “mickey_ears” folder and directly uploaded the images to the “mickey_ears” folder. The opensfm command was returning 0 reconstructions, so I thought maybe the images were too symmetric. Then, I looked back at my work from the summer and realized my mistake, but since I already have these images, might as well use them.
mickey image

Here’s the command I ran to generate the .ply file. This took about an hour for 60 images.

opensfm command 1

opensfm command 3

And here’s Mickey!

Sparse cloud (converted to HTML file and opened in browser):

The red, green, and blue dots show the estimated camera location and orientation for each image.

Dense cloud (imported PLY file on MeshLabs):

(You can faintly make out “BISV” “Class of ‘24”)

bisv words

Surprise slightly squashed two Mobius strip hearts on the back
mobius hearts

As you can see, the point cloud looks a bit like a super low-resolution picture where everything is pixelated, but it’s actually a collection of points rendered with colors in 3D.

However, in order to run OpenSFM with BSI (bit-sliced indexing, the data structure I will be using for matrix operations), I still need to be able to build OpenSFM on my computer and later incorporate the build into the Docker image. Thus, allow me to explain the problems I have been encountering.

Building OpenSFM

These past two weeks, I have been having trouble building OpenSFM on my computer, which means that the project is failing to configure the source code to run on my local system (MacOS). The build process also includes the compile time, which is probably the most commonly known part of it.

Earlier in the process, I fixed some issues with mismatched dependencies/version issues. In case you want to build OpenSFM for yourself, I will go through the errors in the order they appear.
#1:

pip install -r requirements

a.

Error:

Solution:

Add .txt to requirements and run

pip install -r requirements.txt

b.

Error:
cython error

Solution:

Cython 3.0.0 has an issue where it’s not compatible with pyyaml 5.4.1, so remove the version number from pyyaml in requirements.txt. If you’re working in the terminal, run

open requirements.txt

to open the text editor for the file.
pyyaml solution

c.

Error:

Solution:

You can also ignore this because you will install opencv with Homebrew later.

#2: export PYTHONPATH=/usr/local/lib/python3.7/site-packages:$PYTHONPATH

In case you don’t know how to find your Python path to the installed packages, look at the output of pip install requirements.txt.

Where it says something like “/Library/Frameworks…site-packages” is where your path is. Your path might not start with “/Library/Frameworks.”

export PYTHONPATH=/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages:$PYTHONPATH

You can include any of these export lines into your .zshrc or .bashrc, or, better yet, include it in your CMakeLists.txt with slightly different syntax (search up SET CMake documentation).

#3: export C_INCLUDE_PATH=/usr/local/include

export CPLUS_INCLUDE_PATH=/usr/local/include

export DYLD_LIBRARY_PATH=$HOME/local/lib64

To find where your libraries are installed by Homebrew, run brew info opencv (or any library you know you installed).
brew install location

The “/usr/local/Cellar” is where mine was installed, so I would run

export C_INCLUDE_PATH=/usr/local/Cellar

export CPLUS_INCLUDE_PATH=/usr/local/Cellar

export DYLD_LIBRARY_PATH=$HOME/local/lib64

#4: python3 setup.py build

a.

Error:
openmp error

Solution:

What you see above is an error due to adding OpenMP as a library, but clang only works with libomp, not libgomp. You need to comment out these lines in opensfm/src/CMakeLists.txt:
openmp solution

And add

if(APPLE)

if(CMAKE_C_COMPILER_ID MATCHES "Clang")

    set(OpenMP_C "${CMAKE_C_COMPILER}" CACHE STRING "" FORCE)

    set(OpenMP_C_FLAGS "-fopenmp=libomp -Wno-unused-command-line-argument" CACHE STRING "" FORCE)

    set(OpenMP_C_LIB_NAMES "libomp" "libgomp" "libiomp5" CACHE STRING "" FORCE)

    set(OpenMP_libomp_LIBRARY ${OpenMP_C_LIB_NAMES} CACHE STRING "" FORCE)

    set(OpenMP_libgomp_LIBRARY ${OpenMP_C_LIB_NAMES} CACHE STRING "" FORCE)

    set(OpenMP_libiomp5_LIBRARY ${OpenMP_C_LIB_NAMES} CACHE STRING "" FORCE)

endif()

if(CMAKE_CXX_COMPILER_ID MATCHES "Clang")

  set(OpenMP_CXX "${CMAKE_CXX_COMPILER}" CACHE STRING "" FORCE)

  set(OpenMP_CXX_FLAGS "-fopenmp=libomp -Wno-unused-command-line-argument" CACHE STRING "" FORCE)

  set(OpenMP_CXX_LIB_NAMES "libomp" "libgomp" "libiomp5" CACHE STRING "" FORCE)

  set(OpenMP_libomp_LIBRARY ${OpenMP_CXX_LIB_NAMES} CACHE STRING "" FORCE)

  set(OpenMP_libgomp_LIBRARY ${OpenMP_CXX_LIB_NAMES} CACHE STRING "" FORCE)

  set(OpenMP_libiomp5_LIBRARY ${OpenMP_CXX_LIB_NAMES} CACHE STRING "" FORCE)

endif()

endif()