Chalcogenide glasses (ChGs) are amorphous semiconductors with disordered structures and excessive number of defects. Because of this they possess a high concentration of recombination centers and traps in the band gap to capture free carriers produced by ionizing radiation (gamma, beta etc.). Moreover, it is expected that neutron irradiation induced atomic displacement and defects would not increase the total number of defects significantly. This paper addresses the effect of neutron irradiation on the structures of ChGs via Raman spectroscopic (442 nm, 100 K) study. The experiments involved gamma filtered and non-filtered neutron irradiation (with constant flux) for different durations (10 min, 1hr, 8hr) of GexSe100-x (x= 20, 30, 40) glass thin films. They were thermally deposited on single crystalline silicon wafer. Ag dots were additionally placed on top of them to serve as a source for Ag, creating in this manner structures which could give idea about both - interaction of the ChG with the radiation and radiation induced Ag diffusion in them. The structure of these glasses is usually characterized by corner-sharing (CS), edge-sharing (ES) Se-Se chains and ethane-like structure (ETH). Under irradiation all the structures except for ETH (which is absent at x = 20) for all compositions, show significant changes for low dose radiation but the changes become less responsive to radiation as the dose increases. CS units decrease stability the least and it is seen that with only neutron irradiation Ge40Se60 and Ge20Se80 become more stable (CS/ES ratio higher). Except for ETH units, Ge30Se70 (closest to stoichiometry with less homopolar bonds) does not show much change in structures but as the number of ETH units are very low, this effect is not of high importance. The Ag diffusion in ChGs for two compositions x = 20, 40 studied using optical microscopy reveals that it is a function of the irradiation dose. It occurs because of the high affinity of Ag towards the ChG, formation of charged defects due to irradiation and the relatively opened structure of the ChG. Reason for this structure is the fact that ChGs contain relatively rigid covalent bonds and weaker Van der Waals interconnections. This structure permits the metal ion diffusion though the formed in it voids and channels. Neutron flux induces more defects in the system and thus it is predicted that diffusion would happen at a higher rate in Ge20Se80. The result suggests that as the dose of radiation increases, Ag diffusion saturates and becomes difficult to observe through optical microscope but in general it is found that in Ge20Se80, Ag diffusion rate is higher due to the high affinity between Ag and Se which drives the metal into the chalcogenide film. Discussion is made based on the results about the irradiation effects in the studied glasses related to their specific structure and Ag accommodation.