Summary:
Modified:
fbcode/fbjava/fb-spark-applications/rl/dqn-preprocessing/src/main/scala/com/facebook/spark/rl/MultiStepTimeline.scala
To have a similar table creation process as:
fbcode/fbjava/fb-spark-applications/rl/dqn-preprocessing/src/main/scala/com/facebook/spark/rl/Timeline.scala
Reviewed By: czxttkl
Differential Revision: D19662523
fbshipit-source-id: 9cfd789d4a0d737ca0d0a54019be332104699717
Summary:
Fix the usage example such that it obtains a reasonable policy.
Pull Request resolved: https://github.com/facebookresearch/ReAgent/pull/196
Test Plan:
Imported from GitHub, without a `Test Plan:` line.
Existing tests pass.
Manually ran `run_end_to_end_test.sh` and it passed.
Reviewed By: kittipatv
Differential Revision: D19078228
Pulled By: czxttkl
fbshipit-source-id: d50f4fd258583635d16b087f324cd75d18931200
Summary: The object mapping library (jackson) doesn't give default value when the key is missing from json.
Reviewed By: econti
Differential Revision: D15447137
fbshipit-source-id: f25b818b1f03941f7f393b694a92f0b6025845c3
Summary: Add `extraFeatureColumns` as an option to make it easier to add new columns for different feature types. User would just have to write out what additional columns they want to include in the timeline operator.
Reviewed By: czxttkl
Differential Revision: D15429253
fbshipit-source-id: e2311042d951b4236b37eb52161f39b182e294cc
Summary: I need this mainly so that we can test making policy output ranking directly.
Reviewed By: czxttkl
Differential Revision: D15342509
fbshipit-source-id: 85c20b6e272292d3a0c1840ee1e9e855e9acbc70
Summary: Episode length at x percentile could be either Long or Double, depending on the percentileFunction being used. When the percentileFunction is `fb_approx_percentile`, `val pct_val = res.getAs[Double]("pct")` would cause `java.lang.Long cannot be cast to java.lang.Double` error because `fb_approx_percentile` returns a Long but not a Double. See https://our.intern.facebook.com/intern/chronos/jobinstance?jobinstanceid=8680643934&smc=chronos_atn_admin_client&offset=0 for a failed job example.
Reviewed By: kittipatv
Differential Revision: D15236953
fbshipit-source-id: 643e5063f07ac12c8017089b5a672235aa9a1db5
Summary: Following D13704487, this diff prevents spark timeline operator from returning NULL value, which could cause problems in fblearner flow.
Reviewed By: kittipatv
Differential Revision: D14453081
fbshipit-source-id: 5a6fe6abc52a7980619dbb848625135568d01843
Summary:
Pull Request resolved: https://github.com/facebookresearch/Horizon/pull/97
Newer version of Spark has `aprrox_percentile`, which is functionally equivalent to `fb_approx_percentile`.
Reviewed By: czxttkl
Differential Revision: D14163262
fbshipit-source-id: 11a3e9650c0ecb971b4db82cd4c946a20e8335a5
Summary: `outlierEpisodeLengthPercentile` can be set to filter episode longer than the given value. If the filtering the outliers results in dropping more episodes than expected, the filtering won't be applied.
Reviewed By: czxttkl
Differential Revision: D13999112
fbshipit-source-id: 6a2cd5ba674838e6d72f4d1d6c81775ce9ec217b
Summary: Returning NULL values breaks hive reader in our fblearner flows. However, empty values are required if adding terminal states to training data. This diff changes the columns values for terminal states from NULL to the corresponding empty data structure ({}, [], "", etc).
Reviewed By: MisterTea
Differential Revision: D13704487
fbshipit-source-id: 8398cb272e756f413f379f2459165a583546e8c8
Summary:
Write a new spark pipeline for supporting multi-step RL.
The test in dataswarm is in D13125518
Reviewed By: kittipatv
Differential Revision: D13049646
fbshipit-source-id: e388ca415de6c227c46439474fec5de7f6ff8220
Summary:
The evaluator has become difficult to work with. This diff does the following:
1. Creates an EvaluationDataPage similar to TrainingDataPage.
2. Separates datasets into training and evaluation. Evaluation datasets are stored in order so we do not need to un-shuffle them.
3. Assumes the evaluator can process all of the data in one shot. This assumption is now true because we don't need to store features in EvaluationDataPage and we don't need to evaluate during training.
4. Split evaluator.py into: evaluator.py, cpe.py, *_estimator.py
The diff creates some TODOs that I'll turn into tasks (likely bootcamp tasks):
1. Write tests for the estimators
Reviewed By: econti
Differential Revision: D13279810
fbshipit-source-id: fb11ac35ccfb785051f272abfcb9a549fa8fbe64
Summary: Remove reward timeline since we compute MC loss at end of epoch.
Reviewed By: kittipatv
Differential Revision: D10528238
fbshipit-source-id: 67c17a50df65ef9a37945e8f578bb884ed83df22
Gary Engler
Zhengxing Chen
Zhengxing Chen
Kittipat Virochsiri
Jason Gauci
Jason Gauci
Edoardo Conti