Summary:
- Use refactored gymrunner for making data batch
- Implement train_and_evaluate_generic
- (partially) fill out discrete_dqn_reporter
- fill out model_manager.train_workflow
- replace test_oss_workflow.py with new test.
- simplify Preprocessor.scala and remove unused params in Timeline.scala
Pull Request resolved: https://github.com/facebookresearch/ReAgent/pull/237
Test Plan: Run workflow in scripts/ci/run_end_to_end_test.sh.
Reviewed By: kittipatv
Differential Revision: D21181946
Pulled By: kaiwenw
fbshipit-source-id: 6b53d25dc87a59bb8d82a68b29339c722f648349
Summary:
- Added code coverage on python & scala
- Updated the license text to `BSD 3-Clause` so that [licensee](https://github.com/licensee/licensee) can detect it properly
Pull Request resolved: https://github.com/facebookresearch/ReAgent/pull/239
Reviewed By: kaiwenw
Differential Revision: D21185688
Pulled By: kittipatv
fbshipit-source-id: 55a8a98891ab3a6e63da2148bc23d623b5b177d9
Summary:
Buckification of Spark projects: This diff is forRl: Dqn-preprocessing app. Using it pom.xml file as a guide, a TARGET file was created for the app.
Background Info:
For all Java projects in fbcode, build will be done with buck. Since mid-2018, there's an in-flight move to get completely onto buck for development and deployment of all internal Java code: See:
https://fb.facebook.com/groups/java.eng/permalink/1626137397434869/
Reviewed By: k21
Differential Revision: D20428514
fbshipit-source-id: cf9cfaaa90e56ccc9b7323a3c6e62b3d71f4a1e8
Summary: Added support to validate input table and a success and failure test
Reviewed By: czxttkl
Differential Revision: D20545896
fbshipit-source-id: 1243a975053a0d77f92e409ad54922ec4f3c5259
Summary:
1. Add an option to validate Spark data pipeline via custom SQL.
2. Remove links to FB internal documentation from public scaladoc.
Reviewed By: czxttkl
Differential Revision: D19873339
fbshipit-source-id: e0ce5058bb0f6bb597ea8c1873b58ae2468e94d8
Summary:
Modified:
fbcode/fbjava/fb-spark-applications/rl/dqn-preprocessing/src/main/scala/com/facebook/spark/rl/MultiStepTimeline.scala
To have a similar table creation process as:
fbcode/fbjava/fb-spark-applications/rl/dqn-preprocessing/src/main/scala/com/facebook/spark/rl/Timeline.scala
Reviewed By: czxttkl
Differential Revision: D19662523
fbshipit-source-id: 9cfd789d4a0d737ca0d0a54019be332104699717
Summary:
Fix the usage example such that it obtains a reasonable policy.
Pull Request resolved: https://github.com/facebookresearch/ReAgent/pull/196
Test Plan:
Imported from GitHub, without a `Test Plan:` line.
Existing tests pass.
Manually ran `run_end_to_end_test.sh` and it passed.
Reviewed By: kittipatv
Differential Revision: D19078228
Pulled By: czxttkl
fbshipit-source-id: d50f4fd258583635d16b087f324cd75d18931200
Summary: The object mapping library (jackson) doesn't give default value when the key is missing from json.
Reviewed By: econti
Differential Revision: D15447137
fbshipit-source-id: f25b818b1f03941f7f393b694a92f0b6025845c3
Summary: Add `extraFeatureColumns` as an option to make it easier to add new columns for different feature types. User would just have to write out what additional columns they want to include in the timeline operator.
Reviewed By: czxttkl
Differential Revision: D15429253
fbshipit-source-id: e2311042d951b4236b37eb52161f39b182e294cc
Summary: I need this mainly so that we can test making policy output ranking directly.
Reviewed By: czxttkl
Differential Revision: D15342509
fbshipit-source-id: 85c20b6e272292d3a0c1840ee1e9e855e9acbc70
Summary: Episode length at x percentile could be either Long or Double, depending on the percentileFunction being used. When the percentileFunction is `fb_approx_percentile`, `val pct_val = res.getAs[Double]("pct")` would cause `java.lang.Long cannot be cast to java.lang.Double` error because `fb_approx_percentile` returns a Long but not a Double. See https://our.intern.facebook.com/intern/chronos/jobinstance?jobinstanceid=8680643934&smc=chronos_atn_admin_client&offset=0 for a failed job example.
Reviewed By: kittipatv
Differential Revision: D15236953
fbshipit-source-id: 643e5063f07ac12c8017089b5a672235aa9a1db5
Summary: Following D13704487, this diff prevents spark timeline operator from returning NULL value, which could cause problems in fblearner flow.
Reviewed By: kittipatv
Differential Revision: D14453081
fbshipit-source-id: 5a6fe6abc52a7980619dbb848625135568d01843
Summary:
Pull Request resolved: https://github.com/facebookresearch/Horizon/pull/97
Newer version of Spark has `aprrox_percentile`, which is functionally equivalent to `fb_approx_percentile`.
Reviewed By: czxttkl
Differential Revision: D14163262
fbshipit-source-id: 11a3e9650c0ecb971b4db82cd4c946a20e8335a5
Summary: `outlierEpisodeLengthPercentile` can be set to filter episode longer than the given value. If the filtering the outliers results in dropping more episodes than expected, the filtering won't be applied.
Reviewed By: czxttkl
Differential Revision: D13999112
fbshipit-source-id: 6a2cd5ba674838e6d72f4d1d6c81775ce9ec217b
Summary: Returning NULL values breaks hive reader in our fblearner flows. However, empty values are required if adding terminal states to training data. This diff changes the columns values for terminal states from NULL to the corresponding empty data structure ({}, [], "", etc).
Reviewed By: MisterTea
Differential Revision: D13704487
fbshipit-source-id: 8398cb272e756f413f379f2459165a583546e8c8
Summary:
Write a new spark pipeline for supporting multi-step RL.
The test in dataswarm is in D13125518
Reviewed By: kittipatv
Differential Revision: D13049646
fbshipit-source-id: e388ca415de6c227c46439474fec5de7f6ff8220
Summary:
The evaluator has become difficult to work with. This diff does the following:
1. Creates an EvaluationDataPage similar to TrainingDataPage.
2. Separates datasets into training and evaluation. Evaluation datasets are stored in order so we do not need to un-shuffle them.
3. Assumes the evaluator can process all of the data in one shot. This assumption is now true because we don't need to store features in EvaluationDataPage and we don't need to evaluate during training.
4. Split evaluator.py into: evaluator.py, cpe.py, *_estimator.py
The diff creates some TODOs that I'll turn into tasks (likely bootcamp tasks):
1. Write tests for the estimators
Reviewed By: econti
Differential Revision: D13279810
fbshipit-source-id: fb11ac35ccfb785051f272abfcb9a549fa8fbe64
Summary: Remove reward timeline since we compute MC loss at end of epoch.
Reviewed By: kittipatv
Differential Revision: D10528238
fbshipit-source-id: 67c17a50df65ef9a37945e8f578bb884ed83df22
Zhengxing Chen
Kaiwen Wang
Kai Wen Wang
Kittipat Virochsiri
Steven Koeber
Tian Tian Metzgar
Gary Engler
Zhengxing Chen
Jason Gauci
Jason Gauci
Edoardo Conti