In my professorial life I am using an agent-based simulation tool called NetLogo to do some research. The tool creates software agents that move and take action on the patches that make up its world. My research involves testing the ability of the agents to map out the world (equivalent to learning a complex task) using Reinforcement Learning (Q-Learning) and employing different strategies, the way different businesses might have different philosophies. Right now we are looking at how fast an agent learns using an exploring strategy, compared with agents that use a satisficing strategy. Satisficing is a concept developed by systems scientist Herbert Simon, and it says that businessess don’t spend the effort and resources necessary to find an optimal solution, but take the first acceptable solution they come across.
So far, this has proven true, in simulations where all agents are writing to a common blackboard that describes current knowledge of the world. But what happens if each agent has their own blackboard, their own map, and only update the common blackboard periodically? That’s the problem we are currently doing science to.
The trouble is, and this is why I am writing this, NetLogo is reluctant to definitively label its patches, and the agents need to know that they are talking about the same patch when they update. When NetLogo updates patches, it does so for all patches, and it does so in random order. One way to identify patches is to assign a number based on the XY coordinate and the size of the world map. Here’s one code snippit, pulled from the NetLogo forum:
ask patches[set plabel pxcor + 221 – (21 * pycor)]
Since the NL world has zero for a center coordinate, this produces both positive and negative numbers, and is not useful as an array entry ID. The method I’ve devised can be used to store information in an array, but at the price of not having consecutive numbers adjacent to one another. My code is:
set tempPatchNum 0
….code code code …
set patchID tempPatchNum
set tempPatchNum (tempPatchNum + 1)
end ;;setup patches
In NL, an ask command steps through all members of the referenced group (patches) in random order. So we will visit every patch, and give it its own ID, we just won’t be visiting them in any sequential pattern. When it comes time to update our Q table, an array of data about the payoff available from a patch, we can safely run down the array, knowing that the fifth item for any agent, just for e.g., will correspond to the same patch for all agents.