LiveCode for FM Guide Ben Lui

My App Has Too Much Data, What Can I Do?

Updated Mar 31, 2021

If you are compiling a FileMaker database that has many hundreds of megabytes of data, or heads into the Gigabyte range, you will have some issues to contend with.

  1. LCFM Native may take a long time (hours) to compile it. If it is in the Gigabyte range, it may not complete at all;
  2. Anything but the very top of the range Android devices may struggle to run the app;
  3. Syncing your app could be extremely slow, depending on how it is set up.

Let's take a look at these issues and consider what can be done to mitigate them.

1) A long time to compile

How long LCFM will take to compile your app depends on how big it is in terms of layouts, how much data it contains, how fast your computer is, and how much free memory you have. We would not at this time recommend trying to compile an app much over 3-400mb, however if you have a fast computer with lots of memory you could be successful up to around 1GB. If your app is 5GB or more, its unlikely ever to complete the compile.

With a really big app, if its mostly data, consider compiling the file without any data first, and downloading data your user needs on first run instead (but see the section on syncing lots of data below).

2) Android devices can't handle your app

If you have a large app mid to low range Android devices are going to struggle and take a long time to load it or it may simply crash on open. This may not be an issue if you are in a position to specify a minimum specification for the device your app must run on. For example if you are building an app for a client, you know the environment it is going to run on, and you know they are happy with high end devices a large app may not be a problem. However if you are deploying an app store app, to the general public, you are unlikely to be able to specify the end user device and a very big app is undesirable. Consider whether you can slim it down, keep some data on the server and only download what is needed when it is needed.

Syncing large amounts of data is slow

You may have issues if you have a lot of data which is exchanged between your app and the server. For example if you decide to build your app without data, and sync it all when your app is first opened, then opening your app could be extremely slow, and the end user may in fact think it has crashed while it is downloading all your data. You can mitigate this by appropriate messaging in the app at points where it is syncing. You can also design your app carefully so that it only asks for the data it needs, when it needs it. It is possible to do per user syncing, or per table, and to ensure the app never needs to sync a 1GB container in one go.

Summary

For very large databases, look at having a high end machine to compile on with LCFM Native, try compiling without the data first, check your end users have good enough devices to run the app, slim it down where possible and sync incrementally, ensuring data is downloaded or uploaded only when it is needed.

Previous Article reLogin Setup
Next Article Failed to login to remote Database