There are a few types of files that you will not be able to store in SharePoint Online. Microsoft provides that list, including a description of the file type, here.
You can find Microsoft’s complete list of SharePoint Online’s software boundaries and limits here.