androidperformancesortingkotlinanko

How to sort 144,000 files by lastModified() and length() in Kotlin within a second?


I have the following ArrayList of File.

    var a  = ArrayList<File>()
    var b  = ArrayList<File>()
    var c  = ArrayList<File>()
    var d  = ArrayList<File>()
    var e  = ArrayList<File>()

Once the application has started the above ArrayList will be filed with more than 144,000 files. The total size all these combined would nearly 3.5 GB. I want to sort them by lastModified() or length() within in a second and update the modified ArrayList into RecyclerView.

For ease of sorting I have made above ArrayList into a Array<ArrayList<File>> as follows :

    val mList  = arrayOf(a,b,c,d,e)

To speed up things I do everything in background thread. Code :

          doAsync {
              mList.forEach{ index ->
                  index.sortByDescending { it.lastModified() }
              }
              activityUiThread {
                  setRecyclerViewAdapter() // Update RecyclerView with new sorted files
              }

          }


Link to the library I used sort files do in background thread : https://github.com/Kotlin/anko

The above code takes nearly 3-5 seconds to execute. I want this to be done within a second. How to solve this issue ? I am doing this for android.

If needed Im ready to change the API to do the background task


Solution

  • I want this to be done within a second

    Short: it is impossible in general case. File System (on Android or on any other OS) can be overloaded, so this operation can pause your application sometimes. Please note this.

    However you can speedup the code by using the following algorithm:

    Please use example below. Please note that it does a lot of parallel IO operations.

    Benefits of this solution:

    suspend fun sortFiles(files: Iterable<File>): List<File> {
        val metadataReadTasks: List<Deferred<FileWithMetadata>> = withContext(Dispatchers.IO) 
     {
            files.map { file ->
                async {
                    FileWithMetadata(file)
                }
            }
        }
        val metadatas: List<FileWithMetadata> = metadataReadTasks.awaitAll()
        return metadatas
            .sorted()
            .map {
                it.file
            }
    }
    private class FileWithMetadata(
        val file: File
    ) : Comparable<FileWithMetadata> {
        private val lastModified = file.lastModified()
        private val length = file.length()
        override fun compareTo(other: FileWithMetadata): Int {
            return when (other.length) {
                this.length -> other.lastModified.compareTo(this.lastModified)
                else -> other.length.compareTo(this.length)
            }
        }
    }