..I haven't seen any paper that actually simulates that instead of just verbally discussing it. Please lmk if you know of any papers that actually simulate their symmetric learning rule. Would be very helpful.
Also, forgot to mention, here we implemented the A2 associative retrieval generalized in Dickinson & Burke without implementing their symmetric learning rule (A1/A1 or A2/A2 means increase in association; A1/A2 or A2/A1 means decrease in association). That rule is SUPER unstable in simulations..
Thanks so much for all these comments!
...animals with no behavioral relearning, dopamine cue response is learned equally fast in reacquisition as acquisition, which is again hard to square away with conditioned inhibition.
..trial of relearning requires there to be an actual increase in lick rate from zero. So, cannot do the analysis on these animals. Also, the ones that do relearn do so in the same number of trials as acquisition, which is hard to square away with conditioned inhibition. Finally, even in the...
Missed this earlier. Yes, we see a bimodal distribution in terms of lick rates. Some show anticipatory lick rates <0.5Hz, which we use as the criterion for "no learning" (this is an extremely low lick rate in this task). We remove them to identify when animals relearn because identifying...
...seemingly encoded in and controlled by OFC.
Yes, our explanation is pretty much in line with "Reacquisition is rapid when animals have learned that reinforced trials signal other reinforced trials", i.e., p(US|US) is high. Rapid reacquisition results from an intact retrospective memory and a high prior of p(US) coming from p(US|US),...
..but our results are certainly not an immediate consequence of MSOP.
..quickly prevents the ability of context to trigger A2 of CSA. Collectively, both effects prevent degradation of CSA-reward association during CSB-reward trials experienced after CSA extinction. We're doing a fuller parameter sweep to test if there're extremes where there can be more separation..
which means that CSB fully retains its A1 state in phase 2 of CSA extinction (CSB-reward trials). This leads to rewards going to A2 during CSB, thereby minimizing CSA A2 with reward A1 overlap, necessary for decaying CSA-reward association. The other is that CSA is omitted in phase 2, which...
..closer to acquisition (but this also reduces reacquisition rate in G1), but regardless of parameters, G3 is essentially the same as G1. There are at least two reasons for this. In G3 CSA extinction phase 1, we omit CSB trials. So, the context-CSB association is fully degraded in this phase...
Solid lines are acquisition and dashed lines are reacquisition. This model includes context, the Dickinson & Burke modifications, and others (as in Holmes et al 2020). Context is crucial to get rapid reacquisition in G1. One can mess around with extreme parameters to make G2 reacquisition...
Hey apologies for the delay. Had some deadlines. Yeah, agree with the context overlap part but the Dickinson & Burke MSOP does not capture our Group3 effects. Here are simulated results from a parameter combo that produces rapid reacquisition in Group 1. Blue is G1, red G2 and green G3...
Thanks :) Definitely interested to see if this replicates across species.
In neuroscience however, those seminal discoveries had almost entirely been ignored, with the literature largely focusing on dopamine and RL. 3/3
... whether understanding (or even testing) the scaling laws proposed by Gallistel (now more than 25 years old) is a worthwhile neuroscience endeavor, and we were trying to explain that scaling laws in general have been a very fruitful area of enquiry across fields, as they constrain mechanisms. 2/3
Definitely not describing our own results as being at par with gravity, haha, though I can see now that you can read it that way. That sentence was in response to questions we got about Gallistel's work cited in the previous paragraph. The questions were about ... 1/3
And to the best of my knowledge, comparator like models don't explain faster reacquisition following standard extinction. A generalized version of our contingency calculation looks very much like a comparator operation and can capture these. We don't get into these in this paper though.
Yes, will respond properly when I get time, but RR in this one requires the compound cue being converted to A2. We intentionally designed our task to avoid any compound stimuli. So, our design is closer to retroactive cue interference and models dependent on compound conditioning don't work.
That said, overall, there is a real (but small) difference in DA reward response across conditions. We haven't done the experiments you suggested and so, I can only speculate. My guess is that this is unlikely to explain the proportional scaling of learning rate by IRI due to its small effect size.
Thanks! Great question. The apparent difference in reward magnitude in the individual animals in Fig 2C is primarily just inter-animal variability. You can see the opposite trend in the examples in panel A here.
This is, in a sense, a memory, in that the experience of G2 leaves an imprint on the brain, but in our model, it is not an associative memory. It's just an increase in the general "threshold" at which evidence for an association is considered real enough to drive conditioned behavior.
The explanation we proffer in the paper is "Although not a core prediction of retrospective extinction, this effect may arise because, having truly unlearned a previous association, G2 may be more hesitant to act on it the next time it appears to be active."
So, the "slowness" in group data is just an averaging artifact. Animals do behave as if they lose the memory.
So, in G2, everything looks identical in reacquisition compared to acquisition in DA, but due to an increase in threshold (more on this below) after G2, some animals never cross the threshold and thereby never reacquire.
We think it's due to DA cue response needing to cross a threshold for behavior to be generated ("is the evidence strong enough for me to act on it?").
We don't explicitly show it here, but we have seen here and in other studies that animals can acquire DA cue responses without translating that to behavioral output.
We also see this effect in dopamine (DA) recordings in the NAc. In G2, group averaged DA cue responses grow as fast during reacquisition as during acquisition (Fig 1P).
This group effect happens because some G2 animals never reacquire (Fig 1K). If you look at the animals that do reacquire, the reacquisition rate is exactly as fast as acquisition (Fig 1L).
