Contribution of Real Estate Activities and Financial/Insurance Industry Activities to the Formation of Gross Domestic Product

This research examines and proves the contribution of real estate activities and financial and insurance industry activities to the formation of Indonesia's Gross Domestic Product in 2000-2017. The issue that will be discussed is about the variables that influenced the formation of Indonesia's Gross Domestic Product in 2000-2017. The purpose of this study is to prove the influence of the contribution of real estate activities and financial and insurance industry activities to the formation of Gross Domestic Products in Indonesia. To analyze and prove the hypothesis an empirical test was carried out in the form of Q Square prediction with Smart PLS 3.1 on the magnitude of the influence of the activity sub-sector. The method used in data collection is a combination of secondary data derived from Asia Development Bank data. The method used to analyze data is the time series data method. The results showed that Real Estate Activities is an independent variable that does not have a positive and significant effect on the formation of a total Gross Domestic Product. It can be seen from the statistical t value of 1,800> from the t table value of 1,964 and this can be proven by the original sample value of 0.073 and the significance of 0.073> 0.05 which means that Real Estate Activities has no positive effect and significant to the formation of total Gross Domestic Product. Variable Financial and Insurance Activities are independent variables also do not have a positive and significant effect on the formation of total Gross Domestic Product. This can be seen from the statistical t value of 0.401> from the t table value of 1.964 and this can be proven by the original sample value of 0.689 and no significance of 0.005 = 0.05 which means that Financial and Insurance Activities have no effect positive and significant towards the formation of total Gross Domestic Product.

Introduction
Prospects and developments Property investment in Indonesia is predicted to experience a rapid increase. The Indonesian Real Estate Association (REI) revealed that the property business growth in Indonesia could still experience an increase of 20% -30%. In Indonesia, still need millions of decent houses. Therefore, there are many investors who seize the opportunity to take a lot of profit, namely by investing in property in strategic locations.
One of the growth factors of property investment in Indonesia is the completion of the election celebration. The realization of investment or direct investment will slow down due to the holding of the 2019 legislative and presidential elections. This is because many entrepreneurs are waiting and delaying their investments. The investment growth cycle always slows down in the political year. Entrepreneurs tend to delay investing while waiting for the results of the election. The election results will influence government policy in the next 5 years, including in terms of business and other economic activities. Because usually after the implementation of the agenda popping up infrastructure development projects. This includes housing construction projects. Its development is seen from other factors that also affect the property business in Indonesia is the number of consumers who are interested in cheap Down Payment (DP). Indirectly, this DP determination also affects property demand. So it is not surprising that developers who offer low-cost DP with all the conveniences. In carrying out investment property, the risk of experiencing bankruptcy or loss is very small. When compared with other types of investments, such as investment stocks or deposits. Besides having a small risk, to conduct property investment prospects in Indonesia also does not require sophisticated knowledge. Only special knowledge is needed to understand it as in other investment In addition, the Financial Services Industry Sector showed stable conditions with intermediation performance at a positive level. Despite the increased pressure on global markets, the risk profile of the financial services sector is generally well managed. The adequacy of capital and liquidity levels of the domestic Financial Services Institution (OJK) contributed to the resilience of the Indonesian economy amid increasing pressure on the financial markets. In the future, the financial services industry will develop with the adoption of Financial Technology (Fintech) and the granting of licenses to operate in 35 financial industry service companies other than banks. This is a fundamental question whether the permit is appropriate. Does the operation of Fintech contribute to the Indonesian economy. The hope can play a major role in improving the Indonesian economy. Besides the development of the insurance industry also played a role in the development of the economy in Indonesia. Researchers test it in the long run so it is interesting to review the existing phenomena. Liu (2019) in his research entitled FinTech and Its Disruption to Financial Institutions concluded that FinTech's activities had an impact on Financial Institutions and also had an impact on economic development. In addition, Idris's research in Malaysia (2019) also showed that there was something positive about the Malaysian economy over the Financial Industry revolution which entered the category of the industrial revolution 4.0. This industrial revolution is the latest technological trend in such a sophisticated manner, which has a major influence on the production process in the manufacturing sector, trade and financial sectors, such as the use of artificial intelligence (AI), e-commerce, big data, finTech, and shared economies. If you pay attention to the presence of the 4.0 industrial revolution in the financial industry, it can be seen in the form of financial technology (FinTech), which has become increasingly popular lately.
Research by Liao et al (2019) concluded that in the Chinese region there was an efficiency in the financial system due to the implementation of Internet Finance. It is also a question whether there are positive impacts or negative impacts, especially on a small scale, also a background that strengthens this research. Besides this research is also to strengthen the research conducted by Bernard (2019) states that a movement that seeks to open access to banking services to the broadest possible extent to the public, especially those who have not yet utilized banking services. Elsheshtawy's research (2019) states that the case in Dubai caused urbanization where, due to the 2008 financial crisis, the city was abandoned and expatriates fled. After the crisis the infrastructure grew well. Fang et al (2019) tested factors that could affect the performance of the Real Estate Investment Trust (REIT) with observations in Pakistan, Malaysia, Thailand, Singapore and Hong Kong from 2008-2018 with 350 observations concluded that the Net Asset Value (NAV) as a proxy for REITs performance while internal factors are dividend yields, net income and size, external factors used are stock indexes, inflation and interest rates. The results of this research help investors and portfolio managers to broaden their market understanding of the influential factors that affect REIT performance. In addition this research was inspired by Holroyd's research (2019) stating that the financial industry and economic growth are driven from the production of digital content, words, images, videos, animations or other information of commercial value for distribution through digital means and impact on national industrial policies or strategies national innovation as a driver of economic development. Research conducted by Li et al (2019) states that the information and communication technology industry can affect the financial industry and also the economic sector through technology diffusion and information transformation. His research uses an input-output table (IO)

Financial Technology (Fintech)
Fintech is an abbreviation for the words 'financial' and 'technology', which means that it is an innovation in the field of financial services (Bernards, 2019 The existence of Fintech greatly influenced the lifestyle of the economic community. The combination of effectiveness and technology has a positive impact on society at large. There are several benefits of having Fintech in the community, the first benefit being that Fintech can foster new developments in the field of technological startup that is mushrooming. This can help expand employment and increase economic growth.

Real Estate Activities
Real Estate is related to a plot of land and its environment along with existing buildings and projects on land (Cepni et al., 2020). Pavesi et al (2020) state that the real estate activities are activities that are part of the gross domestic product in a country and part of the gross regional domestic product in an area. Real estate includes all the resources on it, such as plants, minerals, water, and other immovable objects in the area (Fonseka et al., 2020). Schernthanner et al (2020) state that the real estate can be grouped into two broad categories based on their use, namely Residential real estate, which is real estate used for residences, such as flats, housing, and condominiums. Usually in residential areas there are places for small businesses such as minimarkets and home industries. Commercial real estate is this real estate that is used for commercial purposes to make a profit. For example office buildings, shops, warehouses, bonded zones, hotels, tourism areas where there are buildings.

Financial/Insurance Industry Activities
Industry is a business activity that is part of the economic growth of a country where its activities include the processing of raw materials or semi-finished goods into finished goods that provide added value for profit (Riasanow et al., 2020). Packing, assembling, assembling and repairing businesses are part of the industry. Industrial products are not only in the form of goods, but also in the form of services. Ohinata and Picchio (2020) state that the Financial Services is a term used to refer to services provided by the financial industry. Financial services are also used to refer to organizations that handle management: Investment Banks, Bank Funds, Credit Card Companies, Consumer Finance Companies and Securities and Insurance Companies. The financial and insurance industry is a service that provides various services related to money and investment. Based on a survey of financial services is the largest service with a share of 25% of the market capitalization value based on the S&P 500 survey.

The Formation of Gross Domestic Product
Scholl Schermuly (2020) explain that Gross Domestic Product is everything that is produced by society and business, including workers' salaries. GDP data is also a way to find out which economic sectors are experiencing growth or decline. Davidson et al (2020) state that in the United States, this GDP figure is calculated and announced quarterly by the Bureau of Economic Analysis (BEA), which revises the estimated up or down development of data received throughout the quarter. The amount of GDP compared to the quarter or the previous year. If GDP in the fourth quarter rose by two percent, this means that the country's economy has experienced growth of two percent throughout the fourth quarter. The GDP calculation method that uses income parameters is obtained by adding up the income of all employees, employees, company profits, copyright income, rental income and net interest income. This is called the GDP income method calculation. While other methods can be measured by the expenditure approach calculated by adding up total consumption, investment, state expenditure and net exports.

Method
This study uses a data analysis method using SmartPLS software version 3.0. which is run on computer media. Secondary data on the contribution of real estate activities and financial and insurance industry activities, data on the formation of Gross Domestic Product Indonesia in 2000-2017 sourced from the Asia Development Bank. The measurement model is used to test the validity and reliability, while the structural model is used to test causality (hypothesis testing with predictive models).

Data Descriptive
The descriptive statistics in this study are as follows in Based on the Table 1, can be seen that the average value of the Real Estate Activities sector transaction is USD 170.9 billion with the minimum value of USD 31.9 billion and the highest value of USD 379.8 billion. Financial and insurance activities with an average transaction value of USD billion USD 239.2 billion with the highest value of USD 571.1 billion with the lowest value of USD 64.3 billion. The GDP by industrial (Y) with an average production value of USD 4,684.6 billion and the lowest value of USD 1,389.8 billion with the highest value of USD 9,912.7 billion.
The result of t-statistics value in the table path coefficients is presented in the following Figure 1 as a follows: Figure 1. The overall model with coefficient Vol. 11, No. 3; Published by Sciedu Press 196 ISSN 1923-3981 E-ISSN 1923 Based on the model in the Figure 1, it can be seen that both Real Estate Activities and Financial and Insurance Activities do not play a major role in the total Gross Domestic Product. The Hypothesis test results are in the following table: The statistic result of this research in the following Table 3: Based on the results of the hypothesis test, it can be seen that the Real Estate Activities (X1) is an independent variable that does not have a positive and significant effect on the formation of a total Gross Domestic Product. It can be seen from the statistical t value of 1,800> from the t table value of 1,964 and this can be proven by the original sample value of 0.073 and the significance of 0.073> 0.05 which means that Real Estate Activities (X1) has no positive effect and significant to the formation of total Gross Domestic Product. The Financial and Insurance Activities (X2) variable is an independent variable and does not have a positive and significant effect on the formation of total Gross Domestic Product. This can be seen from the statistical t value of 0.401> from the t table value of 1.964 and this can be proven by the original sample value of 0.689 and no significance or less than 0.005 = 0.05 which means that Financial and Insurance Activities (X1) have no effect positive and significant towards the formation of total Gross Domestic Product.

Predictive Relevance
The result of Predictive Value show in Table 3 as a follows:

Real Estate Activities (X1)
Financial and Insurance Activities (X 2 ) 0.018

GDP by industrial (Y)
Real Estate Activities (X 1 ) 0.369 Source: PLS Output (2019). Table 3 it can be show that there is a direct relationship between each independent variable on the dependent variable.

Determination Coefficient Test Results
The result of Adjusted R2 as a follows: Based on the test results of the coefficient of determination at Table 4 the value of R Square is 0.538 and the Adjusted R Square value is 0.508. Thus, the value of R Square illustrates that all independent consisting of Real Estate Activities (X 1 ) and Financial and Insurance Activities (X 2 ) variables in this study are able to represent the GDP by industrial (Y) as a dependent variable.

Discussion
The construction industry and the financial industry as one of the producers that have forward relations and links with other sectors. The dependency relationship that occurs with other economic sectors is not only static but can also be dynamic. In the sense that if there is a lot of demand for the construction sector for a construction and financial development, it will automatically move the building material/material industry, consulting services and various small/home industries, while the construction products in the form of buildings can move the sectors ahead such as manufacturing, agriculture, and other sectors which is funded by banking and finance . The connection between the construction industry and other sectors is in the form of a link between the use of output from other sectors (backward linkages) and the linkage of providing inputs to other sectors (forward linkages). Inputs in the form of materials, machinery, labor, professional staff, technology, management systems and capital, are produced by sectors including: manufacturing, trade, banking education and insurance. So that the relevant government institutions such as the ministry of industry and trade, the department of education, banking, insurance. While construction output is used by other sectors, namely agriculture, transportation, energy (electricity) and services. Wibowo (2004) states that linkage relationship will build a strong correlation for the mining industry construction with other sectors.
The availability of labor in Indonesia is triggered by the high rate of population growth so that the abundant workforce is characterized by the main characteristics of not having enough expertise and cheap wages. In general, labor-intensive labor intensive policies have become very popular because they are used in government programs/donor agencies with the creation of large workers in the short term and at the same time as equal distribution of income distribution (Wibisono et al., 2019). The construction sector policy also has an impact on the choice of construction work methods that do not place too much emphasis on cost effectiveness and minimal product quality. Indonesia as a developing country/agricultural country has a labor availability in the agricultural sector and at the same time will contribute to the construction sector for workers informal as unskilled laborers. Informal workers in the construction sector are mostly contributed by farmers as laborers/laborers who work on the sidelines of the planting season or when the harvest season fails and act as seasonal laborers. Although the contribution of the construction workforce itself can be lower/higher than the estimate, because The Statistical Board only counts related to construction work while many other sectors are carrying out similar construction work (buildings in the agriculture or mining sector), they are not counted.
The contribution of the real estate sector to economic growth tends to stagnate from year to year. The last three years, the portion of the property sector, which consists of construction and real estate, to gross domestic product is around 13%. The property cycle experienced by Indonesia today is still fairly reasonable. In 2013-2014 the real estate sector in Indonesia experienced a boom and if it could not be pushed it would bubble and threaten the economy. Recommendations made so that the Government and Bank Indonesia (BI) provide a variety of relaxation to the property business in the form of reduced interest rates, tax cuts, additional subsidies, and others to help him get excited again. In April 2020, property sector activities experienced challenges and decreased again due to the Corona Virus .
Based on the Meeting of the Board of Commissioners (RDK) of the Indonesian Financial Services Authority in December 2019 assessing the stability of the financial services sector in a safe condition with intermediation of the financial services sector posted a positive performance and risk profile of the manageable financial services industry. The indicators in the form of bank loans recorded positive growth, financing companies financing receivables increased, investment loans grew double digit, fund raising, bank third party funds (DPK) grew, this showed the positive role of financial service institutions in supporting financing the national economy where funds were collected from the financial services sector utilized by the government for development funding.

Conclusions
Real Estate Activities does not have a positive and significant effect on the formation of a total Gross Domestic Product. The Financial and Insurance Activities variable do not have a positive and significant effect on the formation of total Gross Domestic Product. Considering the importance of the construction sector, it is necessary to identify the main issues affecting the efficiency of the construction sector and take corrective actions to enhance economic growth and development. The contribution of the real estate sector to economic growth tends to stagnate from year to year. Recommendations to the fiscal and monetary policy provide a variety of relaxation to the property business in the form of reduced interest rates, tax cuts, additional subsidies to help him get excited again. For the Indonesian Financial Services Authority the stability of the financial services sector in a safe condition with intermediation of the financial services sector posted a positive performance and risk profile of the manageable financial services industry.


extfieldsofstudy:
Computer Science
Mathematics

text:
Real World Games Look Like Spinning Tops

This paper investigates the geometrical properties of real world games (e.g. Tic-Tac-Toe, Go, StarCraft II). We hypothesise that their geometrical structure resemble a spinning top, with the upright axis representing transitive strength, and the radial axis, which corresponds to the number of cycles that exist at a particular transitive strength, representing the non-transitive dimension. We prove the existence of this geometry for a wide class of real world games, exposing their temporal nature. Additionally, we show that this unique structure also has consequences for learning - it clarifies why populations of strategies are necessary for training of agents, and how population size relates to the structure of the game. Finally, we empirically validate these claims by using a selection of nine real world two-player zero-sum symmetric games, showing 1) the spinning top structure is revealed and can be easily re-constructed by using a new method of Nash clustering to measure the interaction between transitive and cyclical strategy behaviour, and 2) the effect that population size has on the convergence in these games.

Introduction
Game theory has been used as a formal framework to describe and analyse many naturally emerging strategic interactions [Smith, 1982, Harsanyi et al., 1988, Gibbons, 1992, Sigmund, 1993, Morrow, 1994, Jackson, 2008, David and Jon, 2010. It is general enough to describe very complex interactions between agents, including classic real world games like Tic-Tac-Toe, Chess, Go, and modern computer-based games like Quake, DOTA or StarCraft II. Simultaneously, game theory formalisms apply to abstract games that are not necessarily interesting for humans to play, but were created for different purposes. In this paper we ask the following question: Is there a common structure underlying the games that humans find interesting and engaging?
Why is it important to understand the geometry of real world games? Games have been used as benchmarks and challenges for the development of artificial intelligence for decades, starting with Shannon's interest in Chess [Shannon, 1950], through to the first reinforcement learning success in Backgammon [Tesauro, 1995], IBM DeepBlue [Campbell et al., 2002] developed for Chess, and the more recent achievements of AlphaGo  mastering the game of Go, FTW  for Quake III: Capture the Flag, AlphaStar [Vinyals et al., 2019] for StarCraft II, OpenAI Five [OpenAI et al., 2019] for DOTA 2, and Pluribus [Brown and Sandholm, 2019] for no-limit Texas Hold 'Em Poker. We argue that grasping any common structures to these real world games is essential to understand why specific solution methods work, and can additionally provide us with tools to develop AI based on a deeper understanding of the scope and limits of solutions to previously tackled problems. The analysis of non-transitive behaviour has been critical for algorithm development in general game theoretic settings in the past Lanctot et al. [2017], Balduzzi et al. [2018Balduzzi et al. [ , 2019.

Agent s t rying t o lose
Nash of t he game Non-t ransit ive cyclic dimensions Non-t ransit ivit y gradually disappears (Sect ion 2) Game geomet ry Transit ive dimension Non-t ransit ive dimension e.g. lengt h of longest cycle Game profile Figure 1: High-level visualisation of the geometry of Games of Skill. It shows a strong transitive dimension, that is accompanied by the highly cyclic dimensions, which gradually diminishes as skill grows towards the Nash Equilibrium (upward), and diminishes as skill evolves towards the worst possible strategies (downward). The simplest example of non-transitive behaviour is a cycle of length 3 that one finds e.g. in the Rock Paper Scissors game. Therefore a good tool to have would be the formalisation of non-transitive behaviour in real world games and a method of dealing with notion of transitive progress built on top of it.
We propose the Game of Skill hypothesis (Fig. 1) and argue that strategies in Games of Skill exhibit a geometry that resembles a spinning top, where the upright axis represents the transitive strength and the radial axis corresponds to cyclic, non-transitive dynamics. We focus on two aspects. Firstly, we theoretically and empirically validate whether the Games of Skill geometry materialises in real world games. Secondly, we unpack some of the key practical consequences of the hypothesis, in particular investigating the implications for training agents. Some of the above listed projects use multi-agent training techniques that are not guaranteed to work in all games. In fact, there are conceptually simple, yet surprisingly difficult cyclic games that cannot be solved by these techniques [Balduzzi et al., 2019]. This suggests that real world games form a subclass of games that is strictly smaller than 2-player symmetric zero-sum (games one side wins while the other side loses e.g. Go, Chess, 1v1 StarCraft, 1v1 DOTA etc.) that is often used as a formalisation. The Game of Skill hypothesis provides such a class, and makes specific predictions about how strategies behave. One clear prediction is the existence of tremendously long cycles, which permeate throughout the space of relatively weak strategies in each such game. Theorem 1 proves the existence of long cycles in a rich class of real world games that includes all the examples above. Additionally, we perform empirical analysis of nine real world games, and establish that the hypothesised Games of Skill geometry is indeed observed in each of them.
Finally, we analyse the implications of the Game of Skill hypothesis for learning. In many of the projects tackling real world games , Vinyals et al. [2019], OpenAI et al. [2019] some form of population-based training Jaderberg et al. [2017], Lanctot et al. [2017] is used, where a collection of agents is gathered and trained against. We establish theorems connecting population size and diversity with transitive improvement guarantees, underlining the importance of population-based training techniques used in many of the games-related research above, as well as the notion of diversity seeking behaviours. We also confirm these with simple learning experiments over empirical games coming from nine real world games.
In summary, our contributions are three-fold: i) we define a game class that models real world games, including those studied in recent AI breakthroughs (e.g. Go, StarCraft II, DOTA 2); ii) we show both theoretically and empirically that a spinning top geometry can be observed; iii) we provide theoretical arguments that elucidate why specific state-of-the-art algorithms lead to consistent improvements in such games, with an outlook on developing new population-based training methods.
Proofs of propositions are provided in Supplementary Materials B, together with details on implementations of empirical experiments (E, G, H), additional data (F), and algorithms used (A, C, D, I, J).

Game of Skill hypothesis
We argue that real world games have two critical features that make them Games of Skill. The first feature is the notion of progress. Players that regularly practice need to have a sense that they will improve and start beating less experienced players. This is a very natural property to keep people engaged, as there is a notion of skill involved. From a game theory perspective, this translates to a strong transitive component of the underlying game structure.
A game of pure Rock Paper Scissors (RPS) does not follow this principle and humans essentially never play it in a standalone fashion as a means of measuring strategic skill (without at least knowing the identity of their opponent and having some sense of their opponents previous strategies or biases).
The second feature is the availability of diverse game styles. A game is interesting if there are many qualitatively different strategies [Deterding, 2015, Lazzaro, 2009, Wang and Sun, 2011, that have their own strengths and weaknesses, whilst on average performing on a similar level in the population. Examples include the various openings in games like Chess and Go, which do not provide a universal advantage against all opponents, but rather work well against other specific openings. It follows that players with approximately the same transitive skill level, can still have imbalanced win rates against specific individuals within the group -their strategies and game styles will counter one another. This creates interesting dynamics, providing players, especially at lower levels of skill, direct information on where they can improve. Crucially, this richness gradually disappears as players get stronger, so at the highest level of play, the outcome relies mostly on skill and less on game style. From a game theory perspective, this translates to non-transitive components that rapidly decrease in magnitude relative to the transitive component as skill improves.
These two features combined would lead to a cone-like shape of the game geometry, with a wide, highly cyclic base, and a narrow top of highly skilled strategies. However, while players usually play the game to win, the strategy space includes many strategies whose goal is to lose. While there is often an asymmetry between seeking wins and losses (it is often easier to lose than it is to win), the overall geometry will be analogous -with very few strategies that lose against every other strategy, thus creating a peaky shape at the bottom of our hypothesised geometry. This leads to a spinning top ( Figure 1) -a geometry, where, as we travel across the transitive dimension, the non-transitivity first rapidly increases, and then, after reaching a potentially huge quantity (more formally detailed later), quickly reduces as we approach the strongest strategies. We refer to games that exhibit such underlying geometry as Games of Skill.

Preliminaries
We first establish preliminaries related to game theory and assumptions made herein. We refer to the options available to any player of the game as a strategy, in the game-theoretic sense Moreover, we focus on finite normal-form games (i.e. wherein the outcomes of a game are represented as a payoff tensor), unless otherwise stated.
We use Π to denote the set of all strategies in a given game, with π i ∈ Π denoting a single pure strategy. We further focus on symmetric, deterministic, zero sum games, where the payoff (outcome of a game) is denoted by f (π i , π j ) = −f (π j , π i ) ∈ [−1, 1]. We say that π i beats π j when f (π i , π j ) > 0, draws when f (π i , π j ) = 0 and loses otherwise. For games which are not fully symmetric (e.g. all turn based games) we symmetrise them by considering a game we play once as player 1 and once as player 2. Many games we talk about have an underlying time-dependent structure; thus, it might be more natural to think about them in the so-called extensive-form, wherein player decision-points are expressed in a temporal manner. To simplify our analysis, we conduct our analysis by casting all such games to the normal-form, though we still exploit some of the time-dependent characteristics. Consequently, when we refer to a specific game (e.g. Tic-Tac-Toe), we also analyse the rules of the game itself, which might provide additional properties and insights into the geometry of the payoffs f . In such situations, we explicitly mention that the property/insight comes from game rules rather than its payoff structure f . This is somewhat different from a typical game theoretical analysis (for normal form games) that might equate game and f . We use a standard tree representation of temporally extended games, where a node represents a state of the game (e.g. the board at any given time in the game of Tic-Tac-Toe), and edges represent what is the next game state when the player takes a specific action (e.g. spaces where a player can mark their × or •). The node is called terminal, when it is an end of the game and it provides an outcome f . In this view a strategy is a deterministic mapping from every state to an action, and an outcome between two strategies is simply the outcome of the terminal state they reach when they play against each other. Figure 2 visualises these views on an exemplary three step game.  Figure 2: Left -extensive form/game tree representation of a simple 3-step game, where in each state a player can choose one of two actions, and after exactly 3 moves one of the players wins. Player 1 takes actions in circle nodes, and player 2 in diamond nodes. Outcomes are presented from the perspective of player 1. Middle -a partial normal form representation of this game, presenting outcomes for 4 strategies, colour coded on the graph representation. Right -a symmetrised version, where two colours denote which strategy one follows as player 1 and which as player 2.
We call a game monotonic when f (π i , π j ) > 0 and f (π j , π k ) > 0 implies f (π i , π k ) > 0. In other words, the relation of one strategy beating another is transitive in the set theory sense. We say that a set of strategies {π i } n i=1 forms a cycle of length n when for each i > 1 we have f (π i+1 , π i ) > 0 and f (π 1 , π n ) > 0. For example, in the game of Rock Paper Scissors we have f (π r , π s ) > 0, f (π s , π p ) > 0, f (π p , π r ) > 0. Arguably no real world game is monotonic or purely cyclic, but rather these two components are present at the same time. There are various ways in which one could define a decomposition of a given game into the transitive and non-transitive components Balduzzi et al. [2019]. In this paper, we establish several notions that could be used to describe these phenomena, depending on some assumptions about the game structure. Specifically, in Section 5, the transitive component corresponds to an index of a layer (cluster of strategies) that we define, while the nontransitivity corresponds to a size of this layer. In Section 6 this notion is relaxed for any game, where the transitive component becomes an index of a newly introduced Nash cluster, and the non-transitivity corresponds to the size of this cluster.
The manner in which we study the geometry of games in this paper is motivated by the structural properties that AI practitioners have exploited to build competent agents for real world games Vinyals et al. [2019], , OpenAI et al. [2019], using reinforcement learning (RL). Specifically, consider an empirical game-theoretic outlook on training of policies in a game (e.g. Tic-Tac-Toe), where each trained policy (e.g. neural network) for a player is considered as a strategy of the empirical game. In other words, an empirical game is a normal-form game wherein AI policies are synonymous with strategies. Each of these policies, when deployed on the true underlying game, yields an outcome (e.g. win/loss) captured by the payoff in the empirical game. Thus, in each step of training, the underlying RL algorithm produces an approximate best response in the actual underlying (n−step, extensive form) game; this approximate best response is then added to the set of policies (strategies) in the empirical game, iteratively expanding it.
This AI training process is also often hierarchical -there is some form of multi-agent scheduling process that selects a set of agents to be beaten at a given iteration (e.g. playing against a previous version of an agent in self-play like fashion , or against some distribution of agents generated in the past Vinyals et al. [2019]), and the underlying RL algorithm used for training new policies performs optimisation to find an agent that satisfies this constraint. Because of this structure, there is a risk that the RL algorithm finds very weak strategies that satisfy the constraint (e.g. strategies that are highly exploitable) or relying on some properties of the opponent behaviour that do not generalise to the wider set of opponents one may be interested in. For example, consider a neural network trained to beat a deterministic policy in Chess, which always opens with the same piece. A policy found by such an approach likely has little chance of generalising to any other opening; even though it might play brilliantly against this particular opponent, it might lose against even the weakest opponents who open differently. Issues like this have been observed in various large-scale projects (e.g. exploits that human players found in the Open AI Five OpenAI et al. [2019] or exploiters in League Training of AlphaStar Vinyals et al. [2019]). This exemplifies some of the challenges of creating AI agents, which are not the same that humans face when they play a specific game. Given these insights, we argue that algorithms can be disproportionately affected by the existence of various non-transitive geometries, in contrast to humans.

Real world games are complex
The spinning top hypothesis implies that at some relatively low level of transitive strength, one should expect very long cycles in any Game of Skill. We now prove, that in a large class of games (ranging from board games such as Go and Chess to modern computer games such as DOTA, StarCraft or first-person games like Quake), one can find tremendously long cycles, as well as any other non-transitive geometries.
We first introduce the notion of n-bit communicative games, which provide a mechanism for lower bounding the number of cyclic strategies. For a given game with payoff f , we define its win-draw-loss version with the same rules and a payoff f † = sign • f , which simply removes the score value, and collapses all wins, draws, and losses onto +1, 0, and -1 respectively. Importantly, this transformation does not affect winning, nor the notion of cycles (though could, for example, change Nash equilibria).
Definition 1. Consider the extensive form view of the win-draw-loss version of any underlying game; the underlying game is called n-bit communicative if each player can transmit n ∈ R + bits of information to the other player before reaching the node whereafter at least one of the outcomes 'win' or 'loss' is not attainable.
For example the game in Figure 2 is 1-bit communicative, as each player can take one out of two actions before their actions would predetermine the outcome. Given this definition we can show that, as games become more communicative, the set of strategies that form non-transitive interactions grows exponentially: Theorem 1. For every game that is at least n-bit communicative, and every antisymmetric win-loss payoff matrix P ∈ {−1, 0, 1} 2 n × 2 n , there exists a set of 2 n pure strategies {π 1 , ..., π 2 n } ⊂ Π such that P ij = f † (π i , π j ), and x = max a∈N a ≤ x.
Proof. Let us assume we are given some P ij . We define corresponding strategies π i such that each starts by transmitting its ID as a binary vector using n bits. Afterwards, strategy π i reads out P ij based on its own id, as well as the decoded ID of an opponent π j , and since we assumed each win-draw-loss outcome can still be reached in a game tree, players then play to win/draw or lose, depending on the value of P ij . We choose π i and π j to follow the first strategy in lexicographic ordering (to deal with partially observable/concurrent move games) over sequences of actions that leads to P ij to guarantee the outcome. Ordering over actions is arbitrary and fixed. Since identities are transmitted using binary codes, there are 2 n possible ones.
In particular, this means that if we pick P to be cyclic -where for each i < 2 n we have P ij = 1 for j < i, P ji = −1 and P ii = 0, and for the last strategy we do the same, apart from making it lose to strategy 1, by putting P 2 n 1 = −1 -we obtain a constructive proof of a cycle of length 2 n , since π 1 beats π 2 n , π 2 n beats π 2 n −1 , π 2 n −1 beats π 2 n −2 , ..., π 2 beats π 1 . In practise, the longest cycles can be much longer (see the example of the Parity Game of Skill in the supplementary materials) and thus the above result should be treated as a lower bound.
Note, that strategies composing these long cycles will be very weak in terms of their transitive performance, but of course not as weak as strategies that actively seek to loose, and thus in the hypothesised geometry they would occupy the thick, middle level of the spinning top. Since such strategies do not particularly target winning or losing, they are unlikely to be executed by a human playing a game. Despite this, we use them to exemplify the most extreme part of the underlying geometry, and given that in both the extremes of very strong and very weak policies we expect non-transitivities to be much smaller than that, we hypothesise that they behave approximately monotonically in both these directions.
Interestingly, for many games, we can compute n by traversing the game tree. The intuition behind this procedure is simple and uses basic recursive reasoning, as detailed below. Remark 1. Consider any fully observable game, where each legal action leads to a different state. All the states that, after taking one action have a determined outcome, are 0-bit communicative. For the remaining ones, we can derive the recursive relation for state s ∈ S, and c(s) being the set of all states to which we can transition by taking an action in s. With this notation, n equals φ of the initial state. In more detail, the logarithm simply measures how many bits of information we can transmit (since the game is fully observable, the opponent perceives the action we took) if we restrict ourselves to selecting an action only from A, which is a subset all possible transitions. Since we are interested in states where we guarantee transmitting a given amount of information, we need to take a minimum over how communicative s ∈ A are. We need to keep track of these quantities independently for each player, and simply remember to take the minimum w.r.t. both communication channels (as we are interested in both players communicating the same amount at each state). The overall time and memory complexity of this method is O(|S|B) where S is the state space and B is the branching factor (the maximum number of actions per state). Details and pseudocode are provided in Supplementary Materials.
Running the above procedure on the game of Tic-Tac-Toe we find it is 5.58-bit communicative (which means that any payoff of size 47 × 47 is realised by some strategies). Additionally, all 1-step games (e.g. RPS) are 0-bit communicative, as all actions immediately prescribe the outcome without the ability to communicate any information.
For games where S is too large to be traversed, we can simply rely on an analogous procedure, but considering some heuristic choice of c (s) ⊂ c(s), thus providing a lower bound on n instead of the exact value. In the game of Go, we can choose c (s) to be playing stones on one half of the board, and show that n ≥ 1000. Proposition 1. The game of Go is at least 1000-bit communicative, thus there exists a cycle of length at least 2 1000 , which is larger than the number of atoms in the observable universe.
Proof. Since Go has a resign action, one can use the entire state space for information encoding, whilst still being able to reach both winning and losing outcomes. The game is played on a 19×19 board -if we split it in half we get 180 places to put stones per side, such that the middle point is still empty, and thus any placement of players stones on their half is legal and no stones die. These 180 fields give each player the ability to transfer 1 i=180 log 2 (i) = log 2 (180!) ≈ 1000 bits. and according to Theorem 1 we thus have a cycle of length 2 1000 > 10 100 . Figure 7 provides visualisation of this construction.
Proposition 2. Modern games, such as StarCraft, DOTA or Quake are n-bit communicative for any n ∈ N, unless a time limit is applied to a game. Even with reasonable time limits of 10 minutes, these games are at least 36000-bit communicative.
The above analysis shows that real world games have an extraordinarily complex structure, which is not commonly analysed in classical game theory. The sequential, n-step aspect of these games makes a substantial difference, as even though one could simply view each of them in a normal form way [Myerson, 2013], this would hide the true structure exposed via our analysis. Naturally, the above does not prove that real world games follow the Games of Skill geometry. To validate the merit of this hypothesis, however, we simply follow the well-established path of proving hypothetical models in natural sciences (e.g. physics). Notably, the rich non-transitive structure (located somewhere in the middle of the transitive dimension) exposed by this analysis is a key property that the hypothesised Game of Skill geometry would imply. More concretely, in Section 8 we conduct empirical game theory-based analysis of a wide range of real world games to show that the hypothesised spinning top geometry can, indeed, be observed.

Layered game geometry
The practical consequences of huge sets of non-transitive strategies are two-fold. First, building naive multi-agent training regimes, that try to deal with non-transitivity by asking agents to form a cycle (e.g. by losing to some opponents), is likely to fail -there are just too many ways in which one can lose without providing any transitive improvement for other agents trained against it. Second, there exists a shared geometry and structure across many games, that we should exploit when designing multi-agent training algorithms. In particular, we show how these properties justify some of the recent training techniques involving population-level play and the League Training used in Vinyals et al. [Vinyals et al., 2019]. In this section, we investigate the implications of such a game geometry on the training of agents, starting with a simplified variant that enables building of intuitions and algorithmic insights.
More intuitively, in these k-layered games, all the non-transitive interaction take place within each layer C i , whilst the skill (or transitive) component of the game corresponds to a layer ID.
Note that for every finite game there exists k ≥ 1 for which it is a k-layered game (though when k = 1 this structure is not useful). Moreover, every monotonic game has as many layers as there are strategies in the game.
Layered games can be challenging for many training algorithms used in practise [OpenAI et al., 2019, such as naive self-play. However, a simple form of fictitious play with a hard limit on population size will converge independently of the oracle used (the oracle being the underlying algorithm that returns a new policy that satisfies a given improvement criterion): Proposition 3. Fixed-size population fictitious play, where at iteration t one replaces some (e.g. the oldest) strategy in Π t with a new strategy π such that ∀ πi∈Πt f (π, π i ) > 0 converges in layered Games of Skill, if the population is not smaller than the size of the lowest layer occupied by at least one strategy in the population |Π t | ≥ |C arg min k :Πt∩C k =∅ | and at least one strategy is above level z. If all strategies are below z, then required population size is that of |C z |.
Intuitively, to guarantee transitive improvements over time it is important to cover all possible strategies. In particular, a population improvement variant of the above procedure, where one requires finding the set of unique strategies that satisfy the same criterion, would guarantee transitive improvement each iteration using an identical proof method. This proposition also leads to a known result of needing just one strategy in the population (e.g. self-play) to keep improving in monotonic games [Balduzzi et al., 2019].
Proposition 3 also shows an important intuition related to how modern AI systems are built -the complexity of the non-transitivity discovery/handling methodology decreases as the overall transitive strength of the population grows. Various agent priors (e.g. search, architectural choices for parametric models such as neural networks, smart initialisation such as imitation learning etc.) will initialise in higher parts of the spinning top, and also restrict the set of representable strategies to the transitively stronger ones. This means that there exists a form of balance between priors one builds into an AI system and the amount of required multi-agent learning complexity required (see Figure 3 for a comparison of various recent state of the art AI systems).
From a practical perspective, there is no simple way of knowing |C z | without traversing the entire game tree. Consequently, this property is not directly transferable to the design of an efficient algorithm (as if one had access to the full game tree traversal, one could simply use Min-Max to solve the game). Instead, this analysis provides an intuitive mechanism, explaining why finite-memory fictitious self-play can work well in practice.  Figure 3: Visualization of various state of the art approaches for solving real world games, with respect to the multi-agent algorithm and agent modules used (on the left). Under the assumption that these projects led to the approximately best agents possible, and that the Game of Skill hypothesis is true for these games, we can predict what part of the spinning top each of them had to explore (represented as intervals on the right). This comes from the complexity of the multi-agent algorithm (the method of dealing with non-transitivity) that was employed -the more complex the algorithm, the larger the region of the top that was likely represented by the strategies using the specific agent stack. This analysis does not expose which approach is better or worse. Instead, it provides intuition into how the development of training pipelines used in the literature enables simplification of nontransitivity avoidance techniques, as it provides an initial set of strategies high enough in the spinning top.

Relaxation of a layered game
In practice, covering all cycles is not feasible, and an assumption of perfect dominance in k-layered games is too strong. Thus, we seek a more tractable notion of transitivity and non-transitivity. In this section, we show that games undergo a natural decomposition related to Nash equilibria, which also describes the Game of Skill hypothesis.
The idea behind this approach, called Nash clustering, is to first find the mixed Nash equilibrium of the game payoff P over the set of pure strategies Π (we use notation Nash(P|X) to denote the equilibrium for payoff P when restricted only to strategies in X), and form a first cluster by taking all the pure strategies in the support of this mixture. Then, we restrict our game to the remaining strategies, repeating the process until no strategies remain. Definition 3 (Nash clustering). We define Nash clustering C of the finite zero-sum symmetric game strategy Π set by setting Since a Nash always exists, each cluster is well defined, and since each Nash of a non-empty game has non-empty support, each cluster has non-zero size until some k where j C j = 1≤j≤k C j = Π. By construction we also have for each i, j : C i ∩ C j = ∅, thus ensuring a valid clustering.
While there might be many Nash clusterings per game, there exists a unique maximum entropy Nash clustering where at each iteration we select a Nash equlibrium with maximum Shannon entropy, which is guaranteed to be unique [Ortiz et al., 2007] due to the convexity of the objective.
Nash clustering induces a form of monotonic clustering, in the sense of Relative Population Performance (RPP) [Balduzzi et al., 2019], which is defined for two sets of agents Π A , Π B with a corresponding Nash equilibrium of the asymmetric game (p A , p B ) := Nash(P AB |(A, B)) as RPP(Π A , Π B ) = p T A · P AB · p B . Theorem 2. Nash clustering satisfies RPP(C i , C j ) ≥ 0 for each j > i.
Proof. By definition for each A and each B ⊂ B we have RPP(A, B ) ≥ RPP(A, B), thus for X i := Π \ k<i C k and every j > i we have C j ⊂ X i and (2) For a k-layered game, Nash clusters do not cross the layers' boundaries, but can split layers into smaller clusters. We view them as a form of generalising layers to arbitrary games by relaxing the notion of transitivity and thus relaxing the notion of k-layered games. In fully transitive games, Nash clustering creates one cluster per strategy, and in games that are fully cyclic (e.g. RPS) it creates a single cluster for the entire strategy set.
As in the previous construction, we show that a diverse population that spans an entire cluster (layer) guarantees transitive improvement, despite not having access to any weaker policies nor knowledge of covering the cluster. Theorem 3. If at any point in time, the training population Π t includes any full Nash cluster C i ⊂ Π t , then training against it by finding π such that ∀ πj ∈Πt f (π, π j ) > 0 guarantees transitive improvement in terms of the Nash clustering ∃ k<i π ∈ C k .
Proof. Lets assume that ∃ k>i π ∈ C k . This means, that where the last inequality comes from the fact that C i ⊂ Π t and ∀ πj ∈Πt f (π, π j ) > 0 implies that ∀ πj ∈Ci f (π, π j ) > 0. This leads to a contradiction with the Nash clustering and thus π ∈ C k for some k ≤ i. Finally π cannot belong to C i itself since ∀ πj ∈Ci f (π, π j ) > 0 = f (π, π).
Consequently, in order to keep improving transitively, it is helpful to seek wide coverage of strategies around the current transitive strength (inside the cluster). This high level idea has been applied to single-player domains, where various notions of diversity were studied [Eysenbach et al., 2018, Pathak et al., 2017, but also in some multi-player games such as soccer [Le et al., 2017] and more recently StarCraft II. AlphaStar [Vinyals et al., 2019] explicitly attempts to cover the non-transitivities using exploiters, which implicitly try to expand on the current Nash. With the Game of Skill geometry one can rely on this required coverage to be smaller over time (as agents get stronger). Thus, forcing the new generation of agents to be the weakest ones that beat the previous one would be sufficient to keep covering cluster after cluster, until reaching the final one.

Random Games of Skill
We show that random games also exhibit a spinning top geometry and provide a possible model for Games of Skill, which admits more detailed theoretical analysis. Definition 4 (Random Game of Skill). We define a payoff of a Random Game of Skill as a random antisymmetric matrix, where each entry equals: The intuition behind this construction is that S i will capture part of the transitive strength of a strategy π i . If all the W ij components were removed then the game would be fully monotonic. It can be seen as a linear version of a common Elo model Elo [1978], where each player is assigned a single ranking, which is used to estimate winning probabilities. On the other hand, W ij is responsible for encoding all interactions that are specific only to π i playing against π j , and thus can represent various non-transitive interactions (i.e. cycles) but due to randomness, can also sometimes become transitive.
Let us first show that the above construction indeed yields a Game of Skill, by taking an instance of this game of size n × n.
Proposition 4. If max i,j |W i,j | < α 2 then the difference between maximal and minimal S i in each Nash cluster C a is bounded by α: First, let us note that as the ratio of σ S to σ W grows, this implies that the number of Nash clusters grows as each of them has upper bounded difference in S i by α that depends on magnitude of σ W , while high value of σ S guarantees that there are strategies π i with big differences in corresponding S i 's. This constitutes of the transitive component of the random game. To see that the clusters sizes are concentrated around zero, lets note that because of the zero-mean assumption of S i , this is where majority of S i 's are sampled from. As a result, there is a higher chance of W ij forming cycles there, then it is in less densely populated regions of S i scale. With these two properties in place . Figure 4 further visualises this geometry. This shape can also be seen by considering the limiting distribution of mean strengths. Proposition 5. As the game size grows, for any given k ∈ [n] the average payoff 1 n n i=1 f (π k , π i ) behaves like N (S k , 2σ 2 n ). Now, let us focus our attention on training in such a game, given access to a uniform improvement oracle, which given a set of m opponents returns a uniformly selected strategy from strategy space, among the ones that beat all of the opponents, we will show probability of improving average transitive strength of our population at time t, denoted asS t . Theorem 4. Given a uniform improvement oracle we have that,S t+1 >S t − W, where W is a random variable of zero mean and variance σ 2 m 4 . Moreover, we have E[S t+1 ] > E[S t ]. The theorem shows that the size of the population, against which we are training, has a strong effect on the probability of transitive improvement, as it reduces the variance of W at a quartic rate. This result concludes our analysis of random Games of Skill, we now follow with empirical confirmation of both the geometry and properties predicted made above.

Real world games
In order to empirically validate the spinning top geometry, we consider a selection of two-player zerosum games available in the OpenSpiel library [Lanctot et al., 2019]. Unfortunately, the strategy space Table 1: Game profiles of empirical game geometries, when sampling strategies in various real world games, such as Connect Four, Tic Tac Toe and even StarCraft II. The first three rows shows clearly the Game of Skill geometry, while the last row shows the geometry for games that are not Games of Skill, and clearly do not follow this geometry. Rows of the payoffs are sorted by mean winrate for easier visual inspection. The pink curve shows a fitted Skewed Gaussian to show the spinning top shape, details provided in Supplementary Materials. is enormous even for the simplest of real world games. For example, the number of behaviourally unique pure strategies in Tic-Tac-Toe is larger than 10 567 (see supplementary materials). A full Table 2: Learning curves in empirical games, using a naive population training method for various population sizes: the oldest strategy in the population is replaced with one that beats the whole population on average using an adversarial oracle (returning the weakest strategy satisfying this goal). For Games of Skill (top) there is a phase change of behaviour for most games, where once the population is big enough to deal with the non transitivity, the system converges to the strongest policy. On the other hand in other games (bottom) like disc game, no population size avoids cycling, and for fully transitive games like Elo game, even naive self play converges.
enumeration-based analysis is therefore computationally infeasible. Instead, we rely on empirical game-theoretic analysis, an experimental paradigm that relies on simulation and sampling of strategies to construct abstracted counterparts of complex underlying games, which are more amenable for analysis [Walsh et al., 2002, 2003, Phelps et al., 2004, Wellman, 2006, Phelps et al., 2007, Tuyls and Parsons, 2007. Specifically, we look for strategy sampling that covers the strategy space as uniformly as possible so that the underlying geometry of the game (as exposed by the empirical counterpart) is minimally biased. A simple and intuitive procedure for strategy sampling is as follows. First, apply a tree-search method, in the form of Alpha-Beta [Newell and Simon, 1976] and MCTS [Brügmann, 1993] and select a range of parameters that control the transitive strength of these algorithms (depth of search for Alpha-Beta and number of simulations for MCTS) to ensure coverage of transitive dimension. Second, for each such strategy we create multiple instances, with varied random number seed, thus causing them to behave differently. We additionally include Alpha-Beta agents that actively seek to lose, to ensure discovery of the lower cone of the hypothesised spinning top geometry. While this procedure does not guarantee uniform sampling of strategies, it at least provides decent coverage of the transitive dimension. In total, this yields approximately 1000 agents per game. Finally, following strategy sampling, we form an empirical payoff table with entries evaluating the payoffs of all strategy match-ups, remove all duplicate agents, and use this matrix to approximate the underlying game of interest. Table 1 summarises the empirical analysis which, for the sake of completeness, includes both Games of Skill and games that are not Games of Skill such as for example the Disc game [Balduzzi et al., 2019], a purely transitive Elo game, and the Blotto game. Overall, all real world games results show the hypothesised spinning top geometry. If we look at e.g. Go (3x3) we notice that Nash clusters induced payoff look monotonic, and the sizes of these are maximal around the middle of the transitive strength, and quickly decrease as transitive strength both increases or decreases. At the level of the strongest strategies, we still have non-trivial Nash clusters, showing that even in this empirical approximation of the game of Go on a small board, one still needs some diversity of play styles. This is to be expected due to various game symmetries of the game rules. At the same time various games that were created to study game theory rather than for humans to play fail to exhibit the hypothesised geometry. If we look at Blotto, we see that the size of Nash clusters keep increasing, as the number of strategies one needs to mix at higher and higher level of play in this game keeps growing. This is a desired property for the purpose of studying complexity of games, but arguably not a desired property for a game that is simply played for enjoyment. In particular, the game of Blotto requires players to mix uniformly over all possible permutations to be unexploitable (since the game is invariant to permutations), which is a very hard thing to do for a human player.
We tested the population size claims of Nash coverage as follows. First, construct empirical games coming from the sampling of n agents defined above. This acts as an approximation of the underlying games. Second, define a simple learning algorithm, where we start with k (size of the population) weakest strategies (wrt. mean win-rate) and iteratively replace the oldest one with a strategy π that beats the whole population Π t on average, meaning that π ∈Πt f (π, π ) > 0. To pick a strategy we use the most pessimistic oracle, which selects the weakest strategy satisfying the win-rate condition. This counters the bias towards sampling stronger strategies. As a result, we hope to get a more fair approximation of typical greedy learning methods such as gradient-based methods or reinforcement learning.
For small population sizes training does not converge and cycles for all games ( Table 2). As the population grows, strength increases but saturates in various suboptimal cycles. However, when the population exceeds a critical size, training converges to the best strategies in almost all experiments. For games that are not real world games we observe quite different behaviour -where despite growth of population size cycling keeps occuring (e.g. Disc game), convergence is guaranteed even with a population of size 1 (Elo game which is monotonic).

Conclusions
In this paper we have introduced Games of Skill, a class of games that, as shown both theoretically and empirically, includes many real world games, including Tic Tac Toe, Chess, Go and even StarCraft II and DOTA. In particular we showed, that n-step games have tremendously long cycles, and provided both mathematical and algorithmic methods to estimate this quantity. We showed, that Games of Skill have a geometry resembling a spinning top, which can be used to reason about their learning dynamics. In particular, our insights provide useful guidance for research into population-based learning techniques building on League training [Vinyals et al., 2019] and PBT , especially when enriched with notions of diversity seeking [Balduzzi et al., 2019]. Interestingly, we show that many games from classical game theory are not Games of Skill, and as such might provide challenges that are not necessarily relevant to developing AI methods for real world games. We hope that this work will encourage researchers to study real world games structures, to build better AI techniques that can exploit their unique geometries.

Algorithm 1 Solver for Eq. 4 in O(|B|).
Input: functions f , g and set B:

A Computing n in n-bit communicative games
Our goal is to be able to encode identity of a pure strategy in actions it is taking, in such a way, that opponent will be able to decode it. We focus on fully observable, turn-based games. Note, that with pure policy, and fully observable game, the only way to sent information to the other player is by taking an action (which is observed). Consequently, if at given state one considers A actions, then choosing one of them we can transmit log 2 (A) bits. We will build our argument recursively, by considering subtrees of a game tree. Naturally, a subtree is a tree of some game. Since the assumption of n-bit communicativeness is that we can transmit n bits of information before outcomes become independent, it is easy to note that a subtree for which we cannot find terminal nodes with both outcomes (-1, +1) is 0-bit communicative. Let's remove these nodes from the tree. In the new tree, all the leaf nodes are still 0-bit communicative, as now they are "one action away" from making the outcome deterministic. Let's define function φ per state, that will output how many bits each player can transmit, before the game becomes deterministic, so for each player j φ j (s) = 0 if s is a leaf.
The crucial element is how to now deal with a decision node. Let's use notation c(s) to denote set of all children states, which we assume correspond to taking actions available in this state. If many actions would lead to the same state, we just pretend only one such action exists. From the perspective of player j, what we can do, is to select a subset of states that are reachable from s. If we do so, we will be able to encode log 2 |c(s)| bits in this move plus whatever we can encode in the future, which is simply min s ∈c(s) φ j (s ) as we need to guarantee being able to transmit this number of bits no matter which path is taken. φ j (s) = max I⊂c(s) {log 2 |c(s)| + min s ∈c(s) φ j (s )} However, our argument is symmetric, meaning that we need to not only transmit bits as player j, but also our opponent, and to do so we need to consider minimum over players respective communication channels: It is easy to notice that for a starting state s 0 we now have that the game is min i φ i (s 0 )-bit communicative. The last recursive equation might look intractable, due to iteration over subsets of children states. However, we can easily compute quantities like this in linear time. Let's take general form of and let's consider Alg. 1. To prove that it outputs maximum of g, let's assume that at any point t we decided to pick b = b t . Since b t has highest g at this point, we have g 1 (b ) < g 1 (b t ), and consequently g(C t−1 ∪ {b }) < g(C t−1 ∪ {b t }) so we decreased function value and conclude optimality proof.
We provide a pseudocode in Alg. 2 for the two-player, turn-based case with deterministic transitions. Analogous construction will work for k players, simultaneous move games, as well as games with chance nodes (one just needs to define what we want to happen there, taking minimum will guarantee transmission of bits, and taking expectation will compute expected number of bits instead).
Exemplary execution at some state of Tic-Tac-Toe is provided in Figure 6. Figure 7 shows the construction from Proposition 1 for the game of Go.
We can use exactly the same procedure to compute n-communicativeness over restricted set of policies. For example let us consider strategies using MinMax algorithm to a fixed depth, between 0 and 9. Furthermore, we restrict what kind of first move they can make (e.g. only in the centre, or in a way that is rotationally invariant). Each such class simply defines a new "leaf" labelling of our tree or set of available children. Once we reach a state, after which considered policy is deterministic, by definition its communicativeness is zero, so we put φ(s) = 0 there. Then we again run the recursive procedure. Running this analysis on the game of Tic-Tac-Toe (Fig. 5) reveals the Spinning Top like geometry wrt. class of policies used. As MinMax depth grows, cycle length bound from Theorem 1 decreases rapidly. Similarly introducing more inductive bias in the form of selecting what are good first moves affect the shape in an analogous way. This example has two important properties. First, it Figure 5: Visualisation of cycle bound lengths coming from Theorem 1, when applied to the game of Tic-Tac-Toe over restricted set of policies -y axis corresponds to the depth of MinMax search (encoding transitive strength); and colour and line style correspond to restricted first move (encoding better and better inductive prior over how to play this game).
shows cyclic dimensions behaviour over whole policy space, as we do not rely on any sampling, but rather consider the whole space, restricting the transitive strength and using Theorem 1 as a proxy of non-transitivity. Second, it acts as an exemplification of the claim of various inductive biases restricting the part of the spinning top one needs to deal with when developing and AI for the specific game.

B Proofs
Proposition 2. Modern games, such as StarCraft, DOTA or Quake are n-bit communicative for any n ∈ N, unless time limit is applied. Even with reasonable time limits of tens of minutes, these games are at least 36000-bits communicative.
Proof. With modern games running at 60Hz, as long as agents can "meet" in some place, and execute 60 actions per second that does not change their visibility (such as tiny rotations), they can transmit 60 · 60 · 10 = 36000 bits of information per 10 minute encounter. Note, that this is very loose lower bound, as we are only transmitting one bit of information per action, while this could be significantly enriched, if we allow for use of multiple actions (such as jumping, moving multiple units etc.).
Proposition 3. Fixed-size population fictitious play, where at iteration t one replaces some (e.g. the oldest) strategy in Π t with a new strategy π such that ∀ πi∈Πt f (π, π i ) > 0 converges in layered games of skill, if the population is not smaller than the size of the lowest layer occupied by at least one strategy in the population |Π t | ≥ |C arg min k :Πt∩C k =∅ | and at least one strategy is above level z. If all strategies are below z, then required population size is that of |C z |.
Proof. Let's assume at least one strategy is above z. We will prove, that there will be at most |Π t | − 1 consecutive iterations where algorithm will not improve transitively (defined as a new strategy being part of C i where i is smaller than the lowest number of all C j that have non empty intersections with Algorithm 2 Main algorithm to compute n for which a given fully observable two-player zero-sum game is n-bit communicative. Input: Game tree encoded with: -states: s i ∈ S -value of a state: v(s i ) ∈ {−1, 0, +1, ∅} -set of children states c(s i ) ⊂ S -set of parent states d(s i ) ⊂ S -which player moves p(s i ) ∈ {0, 1} begin {Remove states with deterministic outcomes} {Enqueue a parent if all its children were analysed} end if end for end while return min φ(s 0 ) Algorithm 3 Aggregate (Agg) -helper function for Alg. 2 {other player bits} sort m in decreasing order {Order by decreasing communicativeness} order o in the same order b ← (0, 0) Since we require the new strategy π t+1 added at time t + 1 to beat all previous strategies, it has to occupy at least a level, that is occupied by the strongest strategy in Π t . Let's denote this level by C k , then π t+1 improves transitively, meaning that there exists i < k such that π t+1 ∈ C i , or it belongs to C k itself. Since by construction |C k | ≤ |Π t |, this can happen at most |Π t | − 1 times, as each strategy in Π t ∩ C k needs to be beaten by π t+1 and |Π t ∩ C k | < |Π t |. By the analogous argument, if all the strategies are below C z , one can have at most | max i |C i | − 1 consecutive iterations without transitive improvement.
Proposition 4. If max i,j |W i,j | < α 2 then clusters of Nash clustering {C a } have transitive spread bounded by α: Proof. Let us hypothesise otherwise, so we have a Nash with strategy π a and π b such that S a −S b > α. Let us show that π a has to achieve better outcome against each strategy π c than π b f (π a , π c ) − f (π b , π c ) consequently π b cannot be part of the Nash, contradiction.
Furthermore Nashes supports will be highest around 0 transitive strength, where most of the probability mass of S i distribution is centred, and go towards 0 as they go to ±∞.
Proposition 5. As the game size grows, for any given k ∈ [n] the average payoff 1 Proof.
Using the central limit theorem and the fact that E[W ij ] = E[S j ] = 0 , 1 ≤ j ≤ n and that these variables have a variance bounded by σ 2 .
Theorem 4. Given a uniform improvement oracle we have that,S t+1 >S t − W, where W is a random variable of zero mean and variance σ 2 Proof. Uniform improvement oracle, given a set of index of strategies I t ⊂ [n] (the current members of our population) returns an index i t such that, ∀ i∈It f (π it , π i ) > 0 i.e. π it beats any π i , i ∈ I t and creates I t+1 that consists in replacing a randomly picked i ∈ I t by i t . If the oracle cannot return such index then the training process stops. What we care about is the average skill of the population described by I t ,S t := 1 m i∈It S i , where m := |I t |. By the definition of a uniform improvement oracle we have, ∀ i∈It f (π it , π i ) > 0 (6) Thus, if we call a := i t and b is the index of the replaced strategy we get 1 m i∈It+1 = 1 m i∈It whereW ij := 1 2 (W ij − W ji ). This concludes the first part of the theorem. For the second part we notice that since the strategy in I t is replaced uniformly andW ij , 1 ≤ i < j ≤ n are independent of variance bounded by σ 2 , we have, Finally taking the expectation conditioned on I t , we get

C Cycles counting
In general even the problem of deciding if a graph has a simple path of length higher than some (large) k is NP-hard. Consequently we focus our attention only on cycles of length 3 (which embed Rock-Paper-Scissor dynamics). For this problem, we can take adjacency matrix A ij = 1 ⇐⇒ P ij > 0 and simply compute diag(A 3 ), which will give us number of length 3 cycles that pass through each node. Note, that this technique no longer works for longer cycles as diag(A p ) computes number of closed walks instead of closed paths (in other words -nodes could be repeated). For p = 3 these concepts coincide though.

D Nash computation
We use iterative maximum entropy Nash solver for both Nash clustering and RPP Balduzzi et al. [2019] computation. Since we use numerical solvers, the mixtures found are not exactly Nash equilibria. To ensure that they are "good enough" we find a best response, and check if the outcome is bigger than -1e-4. If it fails, we continue iterating until it is satisfied. For the data considered, this procedure always terminated. While usage of maximum entropy Nash might lead to unnecessarily "heavy" tops of the spinning top geometry (since equivalently we could pick smallest entropy ones, which would form more peaky tops) it guaranteed determinisim of all the procedures (as maximum entropy Nash is unique).

E Games/payoffs definition
After construction of each empirical payoff P, we first symmetrise it (so that ordering of players does not matter), and then standarise it P ij := Pij −Pji 2 max |P| for the analysis and plotting to keep all the scales easy to compare. This has no effect on Nashes or transitive strength, and is only used for consistent presentation of the results, as P ∈ [−1, 1] N ×N . For most of the games this was an identity operation (as for most P we had max P = − min P = 1), and was mostly useful for various random games and Blotto, which have wider range of outcomes.

E.2 StarCraft II (AlphaStar)
We use payoff matrix of the League of the AlphaStar Final Vinyals et al. [2019] which represent a big population (900 agents) playing at a wide range of skills, using all 3 races of the game, and playing it without any simplifications. We did not run any of the StarCraft experiments. Sampling of these strategies is least controlled, and comes from a unique way in which AlphaStar system was trained. However, it looks very well aligned with the goal of covering strategy space (thanks to inclusion of what authors call exploiters), and as such fits our analysis well.

E.3 Rock Paper Scissor (RPS)
We use standard Rock-Paper-Scissor payoff of form This game is fully cyclic, and there is no pure strategy Nash (the only Nash-equilibrium is the uniform mixture of strategies).
Maybe surprisingly, people do play RPS competitively, however it is important to note, that in "real-life" the game of RPS is much richer, than its game theoretic counterpart. First, it often involves repeated trials, which means one starts to reason about the strategy opponent is employing, and try to exploit it while not being exploited themselves. Second, identity of the opponent is often known, and since player are humans, they have inherit biases in the form of not being able to play completely randomly, having beliefs, preferences and other properties, that can be analysed (based on historical matches) and exploited. Finally, since the game is often played in a physical environment, there might be various subconscious tells for a given player, that inform the opponent about which move they are going to play, akin to Clever Hans phenomena.

E.4 Disc Game
We use definition of random game from the "Open-ended learning in symmetric zero-sum games" paper Balduzzi et al. [2019]. We first sample N = 1000 points uniformly in the unit circle A i ∼ U (S(0, 1)) and then put Similarly to RPS, this game is fully cyclic.

E.5 Elo game
We sample Elo rating Elo [1978] per player S i ∼ U(0, 2000), and then put P ij := (1 + e −(Si−Sj )/400 ) −1 , which is equivalent of using scaled difference in strength D ij = (S i − S j )/400 squashed through a sigmoid function σ(x) = (1 + e −x ) −1 . It is easy to see that this game is monotonic, meaning that P ij > P jk → P ik . We use N = 1000 samples.

E.6 Noisy Elo games
For a given noise > 0 we first build an Elo game, and then take N 2 independent samples from N (0, ) and add it to corresponding entries of P, creating P . After that, we symmetrise the payoff by putting P := P − P T .

E.7 Random Game of Skill
We put P ij := 1 2 (W ij − W ji ) + S i − S j where each of the random variables W ij , S i comes from N (0, 1). We use N = 1000 samples.

E.8 Blotto
Blotto is a two-player symmetric zero-sum game, where each player selects a way to place N units onto K fields. The outcome of the game is simply number of fields, where a player has more units than the opponent minus the symmetric quantitiy. We choose N=10, K=5, which creates around 1000 pure strategies, but analogous results were obtained for various other setups we tested. One could ask why is Blotto getting more non-transitive as our strength increases. One simple answer is that the game is permutation invariant, and thus forces optimal strategy to be played uniformly over all possible permutations, which makes the Nash support grow. Real world games, on the other hand, are almost always ordered, sequential, in nature.

E.9 Kuhn Poker
Kuhn Poker [Kuhn, 1950] is a two-player, sequential-move, asymmetric game with 12 information states (6 per player). Each player starts the game with 2 chips, antes a single chip to play, then receives a face-down card from a deck of 3 cards. At each information state, each player has the choice of two actions, betting or passing. We use the implementation of this game in the OpenSpiel library [Lanctot et al., 2019]. To construct the empirical payoff matrices, we enumerate all possible policies of each player, noting that some of the enumerated policies of player 1 may yield identical outcomes depending on the policy of player 2, as certain information states may not be reachable by player 1 in such situations. Due to the randomness involved in the card deals, we compute the average payoffs using 100 simulations per pair of policy match-ups for players 1 and 2. This yields an asymmetric payoff matrix (due to sequential-move nature of the game), which we then symmetrise to conduct our subsequent analysis.

E.10 Parity Game of Skill
Let us define a simple n-step game (per player), that has game of skill geometry. It is a two-player, fully-observable, turn based game that lasts at most n-steps. Game state is a single bit s with initial value 0. At each step, player can choose to: 1) flip the bit (a 1 ); 2) guess that bit is equal to 0 (a 2 ); 3) guess the bit is equal to 1 (a 3 ); 4) keep the bit as it is (a 4 ). At (per player) step n the only legal actions are 2) and 3). If any of these two actions is taken, game ends, and a player wins iff it guessed correctly. Since the game is fully observable, there is no real "guessing" here, agents know exactly what is the state, but we use this construction to be able to study the underlying geometry in the easiest way possible. First, we note that this game is n − 1-bit communicative, as at each turn agents can transmit log 2 (|{a 1 , a 3 }|) = 1 bits of information, and game lasts for n steps, and the last one cannot be used to transfer information. According to Theorem 1 this means that every antisymmetric payoff of size 2 n−1×n−1 can be realised. Figure 8 shows that this game with n = 3 has hundreds of cycles, and Nash clusters of size 40, strongly exceeding lower bounds from Theorem 1. Since there are just 161 pure strategies, we do not have to rely on sampling, and we can clearly see Spinning Top like shape in the game profile.
F Other games that are not Games of Skill Table 3 shows a few Noisy Elo Games, which cause Nashes to grow significantly over the transitive dimension. We also run analysis on Kuhn-Poker, with 64 pure policies, which seems to exhibit analogous geometry to Blotto game. Finally, there is also pure Rock Paper Scissor example, with everything degenerating to a single point.

G Empirical Game Strategy Sampling
We use OpenSpiel Lanctot et al. [2019] implementations of AlphaBeta and MCTS players as base of our experiments. We expand AlphaBeta player to MinMax(d, s), which runs AlphaBeta algorithm up till depth d, and if it did not succeed (game is deeper than d) then it executes random action using seed s instead. We also define MaxMin(d, s) which acts in exactly same way, but uses flipped payoff (so seeks to lose). We also include MinMax'(d, s) and MinMax(d, s) which act in the same way as before, but if some branches of the game tree are longer than d, then they are assumed to have value of 0 (in other words these use the value function that is contantly equal to 0). Finally we define MCTS(k, s) which runs k simulations, and randomness is controlled by seed s. With these 3 types of players, we create a set of agents to evaluate of form:  This gives us 2000 pure strategies, that span the transitive axis. Addition of MCTS is motivated by the fact that many of our games are too hard for AlphaBeta with depth 9 to yield strong policies. Also MinMax(0,s) is equivalent to a completely random policy with a seed s, and thus acts as a sort of a baseline for randomly initialised neural networks. Each of players constructed this way codes a pure strategy (as thanks to seeding that act in a deterministic way).

H Empirical Game Payoff computation
For each game and pair of corresponding pure strategies, we play 2 matches, swapping which player goes first. We report payoff which is the average of these two situations, thus effectively we symmetrise games, which are not purely symmetric (due to their turn based nature). After this step, we check if there are any duplicate rows, meaning that two strategies have exactly the same payoff against every other strategy. We remove them from the game, treating this as a side effect of strategy sampling, which does not guarantee uniqueness (e.g. if the game has less than 2000 pure strategies, than naturally we need to sample some multiple times). Consequently each empirical game has a payoff not bigger than 2000 × 2000, and on average they are closer to 1000 × 1000.

I Fitting spinning top profile
For each plot relating mean RPP to size of Nash clusters, we construct a dataset Next, we use Skewed Normal pdf as a parametric model: where φ is a pdf of a standard Gaussian, and Φ its cdf. We further compose this model with simple affine transformation since our targets are not normalised and not guaranteed to equal to 0 in infinities: ψ (x|µ, σ, α, a, b) = aψ(x|µ, σ, α) + b, · and find parameters µ, σ, α, a, b minimising ψ (x i |µ, σ, α, a, b) − y i 2 .
In general, using probability of data under the MLE skewed normal distribution model could be used as a measure of "game of skillness", but its applications and analysis is left for future research.

J Counting pure strategies
For a given 2 player turn-based game we can compute number of behaviourally different pure strategies by traversing the game tree, and again using a recursive argument. Using notation from previous sections, and z j to denote number of pure strategies for player j we put, for each state s such that p(s) = j: z j (s) = 1 if terminal(s), where the second equation comes from the fact, that two pure strategies are behaviourally different if there exists a state, that both reach when facing some opponent, and they take different action there. So to count pure strategies, we simply sum over all our actions, but need to take product of opponent actions that follow, as our strategy needs to be defined in each of possible opponent moves, and each such we multiply in how many ways we can follow from there, completing the recursion. If we now ask our strategies to be able to play as both players (since in turn-based games are asymmetric) we simply report z 1 (s 0 ) · z −1 (s 0 ), since each combination of behaviour as first and second player is a different pure strategy.
For Tic-Tac-Toe z 1 (s 0 ) ≈ 10 124 and z −1 (s 0 ) ≈ 10 443 so in total we have approximately 10 567 pure strategies that are behaviourally different. Note, that behavioural difference does not imply difference in terms of payoff, however difference in payoff implies behavioural difference. Consequently this is an upper bound on number of size of the minimal payoff describing Tic-Tac-Toe as a normal form game.

K Deterministic strategies and neural network based agents
Even though neural network based agents are technically often mixed strategies in the game theory sense (as they involve stochasticity coming either from Monte Carlo Tree Search, or at least from the use of softmax based parametrisation of the policy), in practise they were found to become almost purely deterministic as training progresses Mnih et al. [2016], so modelling them as pure strategies has empirical justification. However, study and extension of presented results to the mixed strategies regime is an important future research direction.  Figure 6: Partial execution of the n-communicativeness algorithm for Tic-Tac-Toe. Black nodes represent states that no longer can reach all possible outcomes. Green ones last states before all the children nodes would be either terminating or are coloured black. The selected children states (building subset A) are encoded in green (for crosses) and blue (for circles), with the edge captioned with number of bits transmitted (logarithm of number of possible children), minimum number of bits one can transmit afterwards, and minimum number of bits for the other player (because it is a turn based game). n at each node is minimum of φ x and φ o , while for a player p making a move in state s, we have φ p = max A⊂c(s) {log 2 |A| + min s ∈A min p φ p (s )}. Red states are the one not selected in the parent node by maximisation over subsets. Go board into regions where black stones (red), and white stones (blue) will play. Each player has 180 possible moves. Centre) Exemplary first 7 moves, intuitively, ordering of stones encodes a permutation over 180, which corresponds to 180 i=1 log 2 (i) = log 2 ( 180 i=1 i) = log 2 (180!) ≈ 1000 bits being transmitted. Right) After exactly 360 moves, board will always look like this, at which point depending on P id(black),id(white) black player will resign (if it is supposed to lose), or play the centre stone (if it is supposed to win).