Coverage for moptipyapps / prodsched / instances.py: 100%

1""" 

2Generate instances for training and testing. 

3 

4In this package, we provide a function for generating instances, i.e., objects 

5of type :class:`~moptipyapps.prodsched.instance.Instance`, in a 

6deterministic fashion for training and testing of MFC scenarios. 

7 

8The function :func:`get_instances` will return a fixed set of instances 

9for a given instance number. It allows you to store and retrieve compatible 

10instance sets of different sizes from a given directory. 

11 

12This is necessary when doing repeatable experiments that average performance 

13metrics over multiple :class:`~moptipyapps.prodsched.instance.Instance` 

14objects. We do not just want to be able to generate the instances, but we also 

15need to store them and to re-load them. Storing them is easy, function 

16:func:`~moptipyapps.prodsched.instance.store_instances` can do it. 

17Loading instances is easy, too, because for that we have function 

18:func:`~moptipyapps.prodsched.instance.load_instances`. 

19 

20However, what do you do if you generated 10 instances in a deterministic 

21fashion, but for your next experiment you only want to use five of them? 

22How do you decide which to use? 

23Or what if you want to use 15 now. How do you make sure that the previous 

24ten instances are part of the set of 15 instances? 

25:func:`get_instances` does all of that for you. 

26It creates the random seeds for the instance creation in the good old 

27deterministic "moptipy" style, using 

28:func:`~moptipy.utils.nputils.rand_seeds_from_str`. 

29It then checks the instance directory for instances to use that comply 

30with the seeds and generates (and stores) additional instances if need be. 

31For this, we use the Thürer-style instance synthesis implemented as 

32:func:`~moptipyapps.prodsched.mfc_generator.sample_mfc_instance`. 

33Thus, we have a consistent way of generating, storing, and loading instances 

34in a transparent way. 

35 

36(The implementation is not overly efficient, but it will do.) 

37 

38>>> from pycommons.io.temp import temp_dir 

39>>> with temp_dir() as td: 

40... inst_1 = get_instances(3, td) 

41... inst_2 = get_instances(1, td) 

42... inst_3 = get_instances(5, td) 

43 

44>>> len(inst_1) 

453 

46>>> len(inst_2) 

471 

48>>> len(inst_3) 

495 

50 

51>>> all(ix in inst_1 for ix in inst_2) 

52True 

53>>> all(ix in inst_3 for ix in inst_1) 

54True 

55""" 

56 

57from typing import Final 

58 

59from moptipy.utils.nputils import rand_seeds_from_str 

60from pycommons.io.path import Path 

61from pycommons.types import check_int_range 

62 

63from moptipyapps.prodsched.instance import ( 

64 Instance, 

65 instance_sort_key, 

66 load_instances, 

67 store_instances, 

68) 

69from moptipyapps.prodsched.mfc_generator import ( 

70 INFO_RAND_SEED, 

71 sample_mfc_instance, 

72) 

73 

74 

75def get_instances(n: int, inst_dir: str) -> tuple[Instance, ...]: 

76 """ 

77 Get the instances for the experiment. 

78 

79 :param n: the expected number of instances 

80 :param inst_dir: the instance directory 

81 """ 

82 check_int_range(n, "n", 1, 1_000_000) 

83 use_dir: Final[Path] = Path(inst_dir) 

84 

85 has_instances: bool = False 

86 if use_dir.exists(): 

87 try: 

88 has_instances = next(use_dir.list_dir(True, False)).endswith( 

89 ".txt") 

90 except StopIteration: 

91 has_instances = False 

92 

93 seeds: Final[list[int]] = rand_seeds_from_str("mfc", n) 

94 insts: Final[list[Instance]] = [] 

95 newly: Final[list[Instance]] = [] 

96 

97 if has_instances: 

98 def __filter(p: Path, seedstrs=tuple(map( 

99 str.casefold, map(hex, seeds)))) -> bool: 

100 """ 

101 Filter the file names. 

102 

103 :param p: the path 

104 :param seedstrs: the internal seed array 

105 :return: `True` if the file name matches, else `False` 

106 """ 

107 return any(map(str.casefold(p.basename()).__contains__, seedstrs)) 

108 

109 for inst in load_instances(use_dir, __filter): 

110 if INFO_RAND_SEED in inst.infos: 

111 seed = int(inst.infos[INFO_RAND_SEED], 16) 

112 if seed in seeds: 

113 insts.append(inst) 

114 seeds.remove(seed) 

115 

116 for seed in seeds: 

117 inst = sample_mfc_instance(seed=seed) 

118 newly.append(inst) 

119 insts.append(inst) 

120 

121 insts.sort(key=instance_sort_key) 

122 if list.__len__(newly) > 0: 

123 newly.sort(key=instance_sort_key) 

124 store_instances(use_dir, newly) 

125 return tuple(insts)