Menu

episode #4: Creating Your Own File Formats

| Sam Bhattacharyya | | 407 | 3198

00:00:00

The idea of having your own file formats seem very strange in 2018, but back in the earlier days of computing, this wasn’t a strange thing at all. Most people are familiar with typical file formats like mp4 for video, mp3 for audio, pdf or docx for documents. Those seem quite fixed and most people have never found weird file formats unless you use specialty software.

file formatscustomizesoftwarezipfile extensionsdotlearncustom files


Introduction

I’m a full stack developer, particularly fond of node.js. Right now the CEO of DotLearn here in Lagos where we have developed a technology called Vectorized Videos to make online learning and video learning fast and affordable in emerging markets. Outside of that I’m just a nerd, just like anyone else.

Early Life

My parents are originally from India, but I was born on a very very tiny island in the Caribbean called Grenada. After the US invasion in 1983, my parents had to evacuate the Island(my dad has a photo of himself handshaking Ronald Reagan). At age 5 my family moved to the US. I studied mechanical engineering in college, I didn’t study anything to do with code or programming because I wanted to work on robots. Specifically, I was fascinated by space robotics like the Mars Rovers.

After college, I started a pHD program in robotics, but fell out of love with it halfway through. So I dropped out of my pHD program and decided to do social work for several years. I joined the US Peace Corps… and moved to a rural village in Mexico. That’s where I got the idea for DotLearn. There was a connectivity issue with online education which made it nearly impossible to take any classes online so I decided to make it free and widely accessible. 

I was living with a host family in Mexico, my host father run this NGO for people with disabilities where he made prosthesis out of scrap metal that he would find around. Given my background in robotics, I started working with him to make 3d printed prosthetic legs and In about a year and a half we had built a full workshop and we had printed 5 prosthetic legs for people. It was something I wanted to run as a full enterprise but I had a purely technical background so I applied to an MBA program and went to MIT for 2 years….

Passions and Aspirations

I like a lot of things, programming is one of my favorite things to do. I want to see this DotLearn project become a success. If and when it becomes a success story, I’d like to go back to the prosthesis project that I mentioned, aside that there are people with other awesome projects that I’d like to support whether with my own time or with my money. On the side I also love studying languages. My wife is Mexican, and I speak Spanish with her all the time. I’ve also been learning Mandarin for about 10 years now and I am at a point where I can fully give my business presentation and pitch in Chinese which may end up being useful if I ever had to talk to Chinese investors (finger crossed).

What is a file, what are file formats and why did you need to create one?

I like this topic because it’s so close to what I’ve been working on for the past few years.

The idea of having your own file formats seem very strange in 2018, but back in the earlier days of computing, this wasn’t a strange thing at all. Most people are familiar with typical file formats like .mp4 for video, .mp3 for audio, .pdf or .docx for documents. Those seem quite fixed and most people have never found weird file formats unless you use specialty software.

So what does it mean to have a .ai file? What does it mean to have a .stl file. 

At the end of the day it seems like this incomprehensible thing where only big organizations or huge companies can create their own file formats. Files are just ways to store information. And a file format is a schema or a pattern for storing information in a certain way in a certain type of file. And so if I open any type of file with an extension, I can read the file and read a certain type of data in a very specific format.

File formats were very common in the earlier days of computing. Nowadays, we’re too used to word software and SAAS platforms that we kind of forget that information from software services need to be stored somewhere. If you’re using the MEAN stack it could be in a MongoDB database hosted somewhere on a cloud server.

If you open a .docx file with the right program on Linux for example, you’ll see that .docx files are just zip files which contain text files with text in xml format. And you’ll find that most file formats in the world are either text files with text written in a specific way or zip files with text files or if you’re dealing with media files (audio, or video) it’s a bit different.

If you go on wikipedia and download a .svg file. I encourage everyone to do that. Open it with a custom program: a text editor like notepad or  gedit, you’ll see a bunch of xml encoded text things. SVG images are text files organized in xml format according to the SVG standard published by W3C. 

Click this link to download an SVG file

Note the file may be saved as .svg.png. If that's the case, rename it to a .svg extension after downloading and ignore all warnings.

If I put a circle tag in xml, I could specify r=50 and color=#f00. It should make a circle with radius 50 that’s red.

It’s just a text file, and you rename it with an extension SVG and image programs are able to load it as an image.

Same with word docs. It’s not substantially different from the way Google drive stores information in MongoDB objects in the cloud. But in this case you’re storing it in a file format on a certain user’s computer.

In the era before web services and databases, a lot of the data was stored offline as file formats. So that’s kind of why in 2018, if you’re new to computers you just assume that file formats are fixed. 

Back in the day it was a necessity.

Our company DotLearn created its own file format .lrn and if you open it up, you’ll see it’s just a zip file with JSON encoded text inside.  Pretty much, any other file format is like that. You have to open it up and view the contents of the file because they are all just text files. And sometimes zip files with text files inside.

Method

File formats are especially useful when you need some kind of data to be stored offline in a standardized way on someone’s device. Examples of companies that have created file formats are people who create video editing software like photoshop or a lot of open source software for audio and video editing. You can save project configurations as files. If you use Webstorm or some kind of IDE, sometimes they create their own file formats to preserve their own data in a way that’s custom to that software. For example, let’s say you wanted to create your own code editor and you wanted to store your users’ configuration in your own config file, you could just name that config file project.myawesomeextension.

If you’ve used web development for example, you’ll see custom file extensions like .yaml.

If you create you own software which needs to store data offline, creating your own file format is the easy way to do it. All you’re just saying is these files are meant to be opened with a specific set of programs.

The caveat is that, just like software, anyone can create one, but no one will find it useful unless people find some kind of underlying use case for it.

Exclusives

Demo at 29min mark

Download Demo File Here (CHM-8012.vvsf)

Controversial Statement at 37min mark

You can comment on any file formats you’ve created and I’ll check it out

If you create you own software which needs to store data offline, creating your own file format is the easy way to do it.
From a marketing perspective, if you need to market your software properly, you could use new file formats as one of the ways.
Files are just ways to store information. And a file format is a schema or a pattern for storing information in a certain way in a certain type of file.

Creating Your Own File Formats by Sam Bhattacharyya

Comments


Please Login to comment