When the request is asked which tokens are have in an intersection, the previous solution was searching for first Index (linearly) and then filtered everything that was coming after that index using `intersect function`
The updated solution will search for the first token index using`binary search`
# Speedup
While this is only one function was updated, the next options were considered:
- [**lin+filter**] old solution
- [**lin+prefix**] old solution with `prefix:wihle` instead of filtering
- [**bin+filter**] binary search with filter
- [**bin+prefix**] binary search with `prefix:wihle` instead of filtering
The speedup highly depends on the file sizes. The bigger/longer files the bigger win is
# Benchmark
## Kickstarter
|lin+filter|lin+prefix|bin+filter|bin+prefix|speedup|
|-|-|-|-|-|
|0.494|0.243|0.390|\***0.117\***| ~4x |
## Swift
|lin+filter|lin+prefix|bin+filter|bin+prefix|speedup|
|-|-|-|-|-|
|1.739|0.740|1.273|\***0.103**\*| ~16x |
## WordPress
|lin+filter|lin+prefix|bin+filter|bin+prefix|speedup|
|-|-|-|-|-|
|1.270|0.526|0.918|0.148| ~8x |
# Testing code
This code was tested with these parts of code (in Release build)
```
fileprivate var counter = 0
fileprivate var times: [String: Double] = [:]
fileprivate let timesQueue = DispatchQueue.init(label: "benchmarks")
fileprivate func timeLog<T>(_ name: String, block: () -> T) -> T {
let start = DispatchTime.now()
let result = block()
let end = DispatchTime.now()
timesQueue.async {
let oldValue = times[name, default:0.0]
let diff = TimeInterval(end.uptimeNanoseconds - start.uptimeNanoseconds) / 1_000_000_000
let newValue = oldValue + diff
times[name] = newValue
counter += 1
if counter % 1000 * times.count == 0 {
print("!!!!: \(times)")
}
}
return result
}
internal func tokens(inByteRange byteRange: NSRange) -> [SyntaxToken] {
let new = timeLog("new") { tokensFast(inByteRange: byteRange) }
let new2 = timeLog("old") { tokensOld(inByteRange: byteRange) }
return arc4random() % 2 == 1 ? new : new2
}
```
* Use binary search when searching for lines indexes
* Update Changelog
* Remove unused old method of searching line by offset
* Update line by offset call with already implemented function
Xcode 11 started using _response files_ [0] to avoid exceeding command
line length limits. For SwiftLint's analyzer rules to find which
compiler arguments are needed for a given file being analyzed, we first
need to expand these response files to their contents.
This was ported over from https://github.com/jpsim/SourceKitten/pull/613
[0]: https://github.com/apple/swift/pull/16362
* Add Mark rule for triple slash Mark comments
* Move changelog entry to appropriate section
* Re-add Package.resolved
* Fix trailing comma
* Rebuild Rules.md using Xcode 10.3
* Combine two regexes into one
Paul Taykalo
JP Simard
Paul Taykalo
Norio Nomura
Marcelo Fabri
MarkoPejovic
Ivan Vavilov
Max Härtwig
Max Härtwig
Nathan Van Fleet
Colton Schlosser