Opis ebooka: Reinforcement Learning
Reinforcement learning (RL) will deliver one of the biggest breakthroughs in AI over the next decade, enabling algorithms to learn from their environment to achieve arbitrary goals. This exciting development avoids constraints found in traditional machine learning (ML) algorithms. This practical book shows data science and AI professionals how to learn by reinforcement and enable a machine to learn by itself.
Author Phil Winder of Winder Research covers everything from basic building blocks to state-of-the-art practices. You'll explore the current state of RL, focus on industrial applications, learn numerous algorithms, and benefit from dedicated chapters on deploying RL solutions to production. This is no cookbook; doesn't shy away from math and expects familiarity with ML.
- Learn what RL is and how the algorithms help solve problems
- Become grounded in RL fundamentals including Markov decision processes, dynamic programming, and temporal difference learning
- Dive deep into a range of value and policy gradient methods
- Apply advanced RL solutions such as meta learning, hierarchical learning, multi-agent, and imitation learning
- Understand cutting-edge deep RL algorithms including Rainbow, PPO, TD3, SAC, and more
- Get practical examples through the accompanying website
Wybrane bestsellery
-
Czy jesteś gotowy, aby wznieść swoje umiejętności programowania w Pythonie na zupełnie nowy poziom? Najnowsza publikacja renomowanego autora Kevina Clarksona to prawdziwa skarbnica wiedzy dla doświadczonych programistów, którzy pragną zgłębić najbardziej zaawansowane aspekty tego wszechstronnego ...
-
Jeśli chcesz po prostu zrozumieć działanie algorytmów, a nie masz ochoty na mozolne przedzieranie się przez setki trudnych stron, to trzymasz w ręku właściwą książkę! Dzięki temu interesującemu, przystępnemu podręcznikowi szybko przyswoisz sobie najważniejsze pojęcia i łatwo zrozumiesz, w jaki sp...(40.20 zł najniższa cena z 30 dni)
40.20 zł
67.00 zł(-40%) -
Unlock the power of C programming to embark on an epic journey of programming expertise with our comprehensive C programming book KEY FEATURES ? Get a solid foundation of C programming by learning the basic principles, including data types, variables, operators, and control structures. ? Hands-...(109.00 zł najniższa cena z 30 dni)
108.90 zł
109.00 zł(-0%) -
Unlock the power of C++ and enhance your algorithmic thinking with this course. From understanding basic operations to mastering complex structures, this course provides a structured pathway for new programmers. Start your coding journey today!
-
This course offers an in-depth introduction to Java and algorithmic thinking, ideal for beginners. Starting with fundamental computer concepts, it progresses through Java programming basics, & algorithmic principles, concluding with file management.
-
Explore the essentials of PHP programming and algorithmic concepts, starting from the basics. This course offers step-by-step guidance to help you understand how computers work, how to use development environments, and more.
-
Delve into the workings of computers, Visual Basic, and IDEs in a structured and beginner-friendly manner. This course covers everything from algorithmic concepts to advanced structures, providing a solid foundation for aspiring programmers.
-
Explore the essentials of C# programming and algorithmic thinking in this beginner-friendly course by Aristides Bouras. Understand how a computer works, delve into development environments, and tackle complex programming challenges step-by-step.
-
Dive into the world of Python programming & algorithmic thinking with this expertly crafted course. Starting from the basics of how a computer works to advanced concepts like object-oriented programming & file manipulation, this course covers it all.
Ebooka "Reinforcement Learning" przeczytasz na:
-
czytnikach Inkbook, Kindle, Pocketbook, Onyx Boox i innych
-
systemach Windows, MacOS i innych
-
systemach Windows, Android, iOS, HarmonyOS
-
na dowolnych urządzeniach i aplikacjach obsługujących formaty: PDF, EPub, Mobi
Masz pytania? Zajrzyj do zakładki Pomoc »
Audiobooka "Reinforcement Learning" posłuchasz:
-
w aplikacji Ebookpoint na Android, iOS, HarmonyOs
-
na systemach Windows, MacOS i innych
-
na dowolnych urządzeniach i aplikacjach obsługujących format MP3 (pliki spakowane w ZIP)
Masz pytania? Zajrzyj do zakładki Pomoc »
Kurs Video "Reinforcement Learning" zobaczysz:
-
w aplikacjach Ebookpoint i Videopoint na Android, iOS, HarmonyOs
-
na systemach Windows, MacOS i innych z dostępem do najnowszej wersji Twojej przeglądarki internetowej
Szczegóły ebooka
- ISBN Ebooka:
- 978-14-920-7234-8, 9781492072348
- Data wydania ebooka:
-
2020-11-06
Data wydania ebooka często jest dniem wprowadzenia tytułu do sprzedaży i może nie być równoznaczna z datą wydania książki papierowej. Dodatkowe informacje możesz znaleźć w darmowym fragmencie. Jeśli masz wątpliwości skontaktuj się z nami sklep@ebookpoint.pl.
- Język publikacji:
- angielski
- Rozmiar pliku ePub:
- 7.8MB
- Rozmiar pliku Mobi:
- 13.2MB
Spis treści ebooka
- Preface
- Objective
- Who Should Read This Book?
- Guiding Principles and Style
- Prerequisites
- Scope and Outline
- Supplementary Materials
- Conventions Used in This Book
- Acronyms
- Mathematical Notation
- Fair Use Policy
- OReilly Online Learning
- How to Contact Us
- Acknowledgments
- 1. Why Reinforcement Learning?
- Why Now?
- Machine Learning
- Reinforcement Learning
- When Should You Use RL?
- RL Applications
- Taxonomy of RL Approaches
- Model-Free or Model-Based
- How Agents Use and Update Their Strategy
- Discrete or Continuous Actions
- Optimization Methods
- Policy Evaluation and Improvement
- Fundamental Concepts in Reinforcement Learning
- The First RL Algorithm
- Value estimation
- Prediction error
- Weight update rule
- The First RL Algorithm
- Is RL the Same as ML?
- Reward and Feedback
- Delayed rewards
- Hindsight
- Reinforcement Learning as a Discipline
- Summary
- Further Reading
- 2. Markov Decision Processes, Dynamic Programming, and Monte Carlo
Methods
- Multi-Arm Bandit Testing
- Reward Engineering
- Policy Evaluation: The Value Function
- Policy Improvement: Choosing the Best Action
- Simulating the Environment
- Running the Experiment
- Improving the -greedy Algorithm
- Markov Decision Processes
- Inventory Control
- Transition table
- Transition graph
- Transition matrix
- Inventory Control
- Inventory Control Simulation
- Multi-Arm Bandit Testing
- Policies and Value Functions
- Discounted Rewards
- Predicting Rewards with the State-Value Function
- Simulation using the state-value function
- Predicting Rewards with the Action-Value Function
- Optimal Policies
- Monte Carlo Policy Generation
- Value Iteration with Dynamic Programming
- Implementing Value Iteration
- Results of Value Iteration
- Summary
- Further Reading
- 3. Temporal-Difference Learning, Q-Learning, and n-Step Algorithms
- Formulation of Temporal-Difference Learning
- Q-Learning
- SARSA
- Q-Learning Versus SARSA
- Case Study: Automatically Scaling Application Containers to Reduce Cost
- Industrial Example: Real-Time Bidding in Advertising
- Defining the MDP
- Results of the Real-Time Bidding Environments
- Further Improvements
- Formulation of Temporal-Difference Learning
- Extensions to Q-Learning
- Double Q-Learning
- Delayed Q-Learning
- Comparing Standard, Double, and Delayed Q-learning
- Opposition Learning
- n-Step Algorithms
- n-Step Algorithms on Grid Environments
- Eligibility Traces
- Extensions to Eligibility Traces
- Watkinss Q( )
- Fuzzy Wipes in Watkinss Q( )
- Speedy Q-Learning
- Accumulating Versus Replacing Eligibility Traces
- Summary
- Further Reading
- 4. Deep Q-Networks
- Deep Learning Architectures
- Fundamentals
- Common Neural Network Architectures
- Deep Learning Frameworks
- Deep Reinforcement Learning
- Deep Learning Architectures
- Deep Q-Learning
- Experience Replay
- Q-Network Clones
- Neural Network Architecture
- Implementing DQN
- Example: DQN on the CartPole Environment
- Why train online?
- Which is better? DQN versus Q-learning
- Case Study: Reducing Energy Usage in Buildings
- Rainbow DQN
- Distributional RL
- Prioritized Experience Replay
- Noisy Nets
- Dueling Networks
- Example: Rainbow DQN on Atari Games
- Results
- Discussion
- Other DQN Improvements
- Improving Exploration
- Improving Rewards
- Learning from Offline Data
- Summary
- Further Reading
- 5. Policy Gradient Methods
- Benefits of Learning a Policy Directly
- How to Calculate the Gradient of a Policy
- Policy Gradient Theorem
- Policy Functions
- Linear Policies
- Logistic policy
- Softmax policy
- Linear Policies
- Arbitrary Policies
- Basic Implementations
- Monte Carlo (REINFORCE)
- Example: REINFORCE on the CartPole environment
- Monte Carlo (REINFORCE)
- REINFORCE with Baseline
- Example: REINFORCE with baseline on the CartPole environment
- Gradient Variance Reduction
- n-Step Actor-Critic and Advantage Actor-Critic (A2C)
- Example: n-step actor-critic on the CartPole environment
- State-value learning decay rates versus policy decay rates
- Eligibility Traces Actor-Critic
- Example: Eligibility trace actor-critic on the CartPole environment
- A Comparison of Basic Policy Gradient Algorithms
- Industrial Example: Automatically Purchasing Products for Customers
- The Environment: Gym-Shopping-Cart
- Expectations
- Results from the Shopping Cart Environment
- Summary
- Further Reading
- 6. Beyond Policy Gradients
- Off-Policy Algorithms
- Importance Sampling
- Behavior and Target Policies
- Off-Policy Q-Learning
- Gradient Temporal-Difference Learning
- Greedy-GQ
- Off-Policy Actor-Critics
- Off-Policy Algorithms
- Deterministic Policy Gradients
- Deterministic Policy Gradients
- Deep Deterministic Policy Gradients
- DDPG derivation
- DDPG implementation
- Twin Delayed DDPG
- Delayed policy updates (DPU)
- Clipped double Q-learning (CDQ)
- Target policy smoothing (TPS)
- TD3 implementation
- Case Study: Recommendations Using Reviews
- Improvements to DPG
- Trust Region Methods
- KullbackLeibler Divergence
- KL divergence experiments
- KullbackLeibler Divergence
- Natural Policy Gradients and Trust Region Policy Optimization
- Proximal Policy Optimization
- PPOs clipped objective
- PPOs value function and exploration objectives
- Example: Using Servos for a Real-Life Reacher
- Experiment Setup
- RL Algorithm Implementation
- Increasing the Complexity of the Algorithm
- Hyperparameter Tuning in a Simulation
- Resulting Policies
- Other Policy Gradient Algorithms
- Retrace( )
- Actor-Critic with Experience Replay (ACER)
- Actor-Critic Using Kronecker-Factored Trust Regions (ACKTR)
- Emphatic Methods
- Extensions to Policy Gradient Algorithms
- Quantile Regression in Policy Gradient Algorithms
- Summary
- Which Algorithm Should I Use?
- A Note on Asynchronous Methods
- Further Reading
- 7. Learning All Possible Policies with
Entropy Methods
- What Is Entropy?
- Maximum Entropy Reinforcement Learning
- Soft Actor-Critic
- SAC Implementation Details and Discrete Action Spaces
- Automatically Adjusting Temperature
- Case Study: Automated Traffic Management to Reduce Queuing
- Extensions to Maximum Entropy Methods
- Other Measures of Entropy (and Ensembles)
- Optimistic Exploration Using the Upper Bound of Double Q-Learning
- Tinkering with Experience Replay
- Soft Policy Gradient
- Soft Q-Learning (and Derivatives)
- Path Consistency Learning
- Performance Comparison: SAC Versus PPO
- How Does Entropy Encourage Exploration?
- How Does the Temperature Parameter Alter Exploration?
- Industrial Example: Learning to Drive with a Remote Control Car
- Description of the Problem
- Minimizing Training Time
- Dramatic Actions
- Hyperparameter Search
- Final Policy
- Further Improvements
- Summary
- Equivalence Between Policy Gradients and Soft Q-Learning
- What Does This Mean For the Future?
- What Does This Mean Now?
- 8. Improving How an Agent Learns
- Rethinking the MDP
- Partially Observable Markov Decision Process
- Predicting the belief state
- Partially Observable Markov Decision Process
- Case Study: Using POMDPs in Autonomous Vehicles
- Contextual Markov Decision Processes
- MDPs with Changing Actions
- Regularized MDPs
- Rethinking the MDP
- Hierarchical Reinforcement Learning
- Naive HRL
- High-Low Hierarchies with Intrinsic Rewards (HIRO)
- Learning Skills and Unsupervised RL
- Using Skills in HRL
- HRL Conclusions
- Multi-Agent Reinforcement Learning
- MARL Frameworks
- Centralized or Decentralized
- Single-Agent Algorithms
- Case Study: Using Single-Agent Decentralized Learning in UAVs
- Centralized Learning, Decentralized Execution
- Decentralized Learning
- Other Combinations
- Challenges of MARL
- MARL Conclusions
- Expert Guidance
- Behavior Cloning
- Imitation RL
- Inverse RL
- Curriculum Learning
- Other Paradigms
- Meta-Learning
- Transfer Learning
- Summary
- Further Reading
- 9. Practical Reinforcement Learning
- The RL Project Life Cycle
- Life Cycle Definition
- Data science life cycle
- Reinforcement learning life cycle
- Life Cycle Definition
- The RL Project Life Cycle
- Problem Definition: What Is an RL Project?
- RL Problems Are Sequential
- RL Problems Are Strategic
- Low-Level RL Indicators
- An entity
- An environment
- A state
- An action
- Quantify success or failure
- Types of Learning
- Online learning
- Offline or batch learning
- Concurrent learning
- Reset-free learning
- RL Engineering and Refinement
- Process
- Environment Engineering
- Implementation
- Simulation
- Interacting with real life
- State Engineering or State Representation Learning
- Learning forward models
- Constraints
- Transformation (dimensionality reduction, autoencoders, and world models)
- Policy Engineering
- Discrete states
- Continuous states
- Converting to discrete states
- Mixed state spaces
- Mapping Policies to Action Spaces
- Binary actions
- Continuous actions
- Hybrid action spaces
- When to perform actions
- Massive action spaces
- Exploration
- Is intrinsic motivation exploration?
- Visitation counts (sampling)
- Information gain (surprise)
- State prediction (curiosity or self-reflection)
- Curious challenges
- Random embeddings (random distillation networks)
- Distance to novelty (episodic curiosity)
- Exploration conclusions
- Reward Engineering
- Reward engineering guidelines
- Reward shaping
- Common rewards
- Reward conclusions
- Summary
- Further Reading
- 10. Operational Reinforcement Learning
- Implementation
- Frameworks
- RL frameworks
- Other frameworks
- Frameworks
- Scaling RL
- Distributed training (Gorila)
- Single-machine training (A3C, PAAC)
- Distributed replay (Ape-X)
- Synchronous distribution (DD-PPO)
- Improving utilization (IMPALA, SEED)
- Scaling conclusions
- Implementation
- Evaluation
- Policy performance measures
- Statistical policy comparisons
- Algorithm performance measures
- Problem-specific performance measures
- Explainability
- Evaluation conclusions
- Deployment
- Goals
- Goals during different phases of development
- Best practices
- Hierarchy of needs
- Goals
- Architecture
- Ancillary Tooling
- Build versus buy
- Monitoring
- Logging and tracing
- Continuous integration and continuous delivery
- Experiment tracking
- Hyperparameter tuning
- Deploying multiple agents
- Deploying policies
- Safety, Security, and Ethics
- Safe RL
- Secure RL
- Ethical RL
- Summary
- Further Reading
- 11. Conclusions and the Future
- Tips and Tricks
- Framing the Problem
- Your Data
- Training
- Evaluation
- Deployment
- Tips and Tricks
- Debugging
- ${ALGORITHM_NAME} Cant Solve ${ENVIRONMENT}!
- Monitoring for Debugging
- The Future of Reinforcement Learning
- RL Market Opportunities
- Future RL and Research Directions
- Research in industry
- Research in academia
- Ethical standards
- Concluding Remarks
- Next Steps
- Now Its Your Turn
- Further Reading
- A. The Gradient of a Logistic Policy for Two Actions
- B. The Gradient of a Softmax Policy
- Glossary
- Acronyms and Common Terms
- Symbols and Notation
- Index
O'Reilly Media - inne książki
-
Large language models (LLMs) and generative AI are rapidly changing the healthcare industry. These technologies have the potential to revolutionize healthcare by improving the efficiency, accuracy, and personalization of care. This practical book shows healthcare leaders, researchers, data scient...(160.65 zł najniższa cena z 30 dni)
160.15 zł
199.00 zł(-20%) -
With hundreds of tools preinstalled, the Kali Linux distribution makes it easier for security professionals to get started with security testing quickly. But with more than 600 tools in its arsenal, Kali Linux can also be overwhelming. The new edition of this practical book covers updates to the ...(174.18 zł najniższa cena z 30 dni)
173.97 zł
209.00 zł(-17%) -
Learn how to implement and manage a modern customer data platform (CDP) through the Salesforce Data Cloud platform. This practical book provides a comprehensive overview that shows architects, administrators, developers, data engineers, and marketers how to ingest, store, and manage real-time cus...(197.34 zł najniższa cena z 30 dni)
197.14 zł
249.00 zł(-21%) -
Real-time applications are becoming the norm today. But building a model that works properly requires real-time data from the source, in-flight stream processing, and low latency serving of its analytics. With this practical book, data engineers, data architects, and data analysts will learn how ...(223.08 zł najniższa cena z 30 dni)
222.88 zł
279.00 zł(-20%) -
Methods of delivering software are constantly evolving in order to increase speed to market without sacrificing reliability and stability. Mastering development end to end, from version control to production, and building production-ready code is now more important than ever. Continuous deploymen...(182.64 zł najniższa cena z 30 dni)
182.58 zł
239.00 zł(-24%) -
As the transformation to hybrid multicloud accelerates, businesses require a structured approach to securing their workloads. Adopting zero trust principles demands a systematic set of practices to deliver secure solutions. Regulated businesses, in particular, demand rigor in the architectural pr...(165.75 zł najniższa cena z 30 dni)
165.65 zł
209.00 zł(-21%) -
This concise yet comprehensive guide explains how to adopt a data lakehouse architecture to implement modern data platforms. It reviews the design considerations, challenges, and best practices for implementing a lakehouse and provides key insights into the ways that using a lakehouse can impact ...(193.19 zł najniższa cena z 30 dni)
192.69 zł
249.00 zł(-23%) -
In today's fast-paced world, more and more organizations require rapid application development with reduced development costs and increased productivity. This practical guide shows application developers how to use PowerApps, Microsoft's no-code/low-code application framework that helps developer...(162.27 zł najniższa cena z 30 dni)
162.17 zł
209.00 zł(-22%) -
Welcome to the systems age, where software professionals are no longer building software&emdash;we're building systems of software. Change is continuously deployed across software ecosystems coordinated by responsive infrastructure. In this world of increasing relational complexity, we need t...(152.01 zł najniższa cena z 30 dni)
151.81 zł
209.00 zł(-27%) -
This book provides an ideal guide for Python developers who want to learn how to build applications with large language models. Authors Olivier Caelen and Marie-Alice Blete cover the main features and benefits of GPT-4 and GPT-3.5 models and explain how they work. You'll also get a step-by-step g...(155.36 zł najniższa cena z 30 dni)
154.86 zł
209.00 zł(-26%)
Dzieki opcji "Druk na żądanie" do sprzedaży wracają tytuły Grupy Helion, które cieszyły sie dużym zainteresowaniem, a których nakład został wyprzedany.
Dla naszych Czytelników wydrukowaliśmy dodatkową pulę egzemplarzy w technice druku cyfrowego.
Co powinieneś wiedzieć o usłudze "Druk na żądanie":
- usługa obejmuje tylko widoczną poniżej listę tytułów, którą na bieżąco aktualizujemy;
- cena książki może być wyższa od początkowej ceny detalicznej, co jest spowodowane kosztami druku cyfrowego (wyższymi niż koszty tradycyjnego druku offsetowego). Obowiązująca cena jest zawsze podawana na stronie WWW książki;
- zawartość książki wraz z dodatkami (płyta CD, DVD) odpowiada jej pierwotnemu wydaniu i jest w pełni komplementarna;
- usługa nie obejmuje książek w kolorze.
Masz pytanie o konkretny tytuł? Napisz do nas: sklep[at]helion.pl.
Książka, którą chcesz zamówić pochodzi z końcówki nakładu. Oznacza to, że mogą się pojawić drobne defekty (otarcia, rysy, zagięcia).
Co powinieneś wiedzieć o usłudze "Końcówka nakładu":
- usługa obejmuje tylko książki oznaczone tagiem "Końcówka nakładu";
- wady o których mowa powyżej nie podlegają reklamacji;
Masz pytanie o konkretny tytuł? Napisz do nas: sklep[at]helion.pl.
Książka drukowana
![Loader](https://static01.helion.com.pl/ebookpoint/img/ajax-loader.gif)
![ajax-loader](https://static01.helion.com.pl/ebookpoint/img/ajax-loader.gif)
Oceny i opinie klientów: Reinforcement Learning Phil Winder Ph. D. (0)
Weryfikacja opinii następuję na podstawie historii zamówień na koncie Użytkownika umieszczającego opinię. Użytkownik mógł otrzymać punkty za opublikowanie opinii uprawniające do uzyskania rabatu w ramach Programu Punktowego.