Keras LSTM模型未产生相同结果

jhkqcmku  于 2023-03-18  发布在  其他
关注(0)|答案(1)|浏览(148)

我已经在文本数据上运行了一个LSTM模型,虽然我已经设置了随机、numpy和tensorflow的种子,但每次仍然没有产生相同的结果。我还设置了内核初始化器的种子,但也没有产生任何结果。我错过了什么吗?(我已经检查了预处理,结果都是一样的)

model = Sequential()
model.add(Embedding(input_dim=vocab_size + 1, input_length=data_max,
                    output_dim=3,
                    embeddings_initializer=keras.initializers.initializers_v2.RandomUniform(seed=seed_value)
                    , mask_zero=True))
model.add(Dropout(rate=0.4, seed=seed_value))
model.add((LSTM(units=4, kernel_initializer=keras.initializers.initializers_v2.GlorotUniform(seed=seed_value))))
model.add(Dropout(rate=0.5, seed=seed_value))
model.add(Dense(units=100, kernel_initializer=keras.initializers.initializers_v2.RandomNormal(0, 1, seed=seed_value)
                , activation='relu'))
model.add(Dropout(rate=0.6, seed=seed_value))
model.add(Dense(units=4, kernel_initializer=keras.initializers.initializers_v2.RandomNormal(0, 1, seed=seed_value)
                , activation='sigmoid'))
model.summary()
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

lstm = model.fit(X_train_vector, Y_train_vector, validation_data=[X_valid, Y_valid], epochs=25, shuffle=False)

这里我展示了如何为numpy、tensorflow和random种子化

# Ensure seed is all the same ------------------
seed_value = 1
os.environ['PYTHONHASHSEED'] = str(seed_value)
random.seed(seed_value)
np.random.seed(seed_value)  # np seed
tf.random.set_seed(seed_value)
session_conf = tf.compat.v1.ConfigProto(
    intra_op_parallelism_threads=1,
    inter_op_parallelism_threads=1)

# Force Tensorflow to use a single thread
sess = tf.compat.v1.Session(graph=tf.compat.v1.get_default_graph(), config=session_conf)
K.set_session(sess)
# ----------------------------------------------
5cnsuln7

5cnsuln71#

您的模型或训练优化器需要学习时间,它将根据优化点进行调整。(问题是为什么相似的随机数种子不能提供相同的结果)

[样品]:

import tensorflow as tf
import numpy as np

"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: Variables
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
learning_rate = 0.001
global_step = 0
tf.compat.v1.disable_eager_execution()

start = 3
limit = 33
delta = 3.0
inputs = tf.range(start, limit, delta)
inputs = tf.expand_dims(inputs, axis=0)
inputs = tf.expand_dims(inputs, axis=0)
lstm = tf.keras.layers.LSTM(4)
output = lstm(inputs)
print(output.shape)
print(output)

"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: Training
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
optimizer = tf.compat.v1.train.ProximalAdagradOptimizer(
    learning_rate,
    initial_accumulator_value=0.1,
    l1_regularization_strength=0.2,
    l2_regularization_strength=0.1,
    use_locking=False,
    name='ProximalAdagrad'
)

var1 = tf.Variable(10.0)
var2 = tf.Variable(10.0)
X_var = tf.compat.v1.get_variable('X', dtype = tf.float32, initializer = tf.random.normal((1, 10, 1)))
y_var = tf.compat.v1.get_variable('Y', dtype = tf.float32, initializer = tf.random.normal((1, 10, 1)))
Z = tf.nn.l2_loss((var1 - X_var) ** 2 + (var2 - y_var) ** 2, name="loss")

cosine_loss = tf.keras.losses.CosineSimilarity(axis=1)
loss = tf.reduce_mean(input_tensor=tf.square(Z))
training_op = optimizer.minimize(cosine_loss(X_var, y_var))

X = np.reshape([ 500, -400, 400, -300, 300, -200, 200, -100, 100, 1 ], (1, 10, 1))
Y = np.reshape([ -400, 400, -300, 300, -200, 200, -100, 100, -50, 50 ], (1, 10, 1))

with tf.compat.v1.Session() as sess:
    sess.run(tf.compat.v1.global_variables_initializer())
    
    for i in range(1000):
        global_step = global_step + 1
        train_loss, temp = sess.run([loss, training_op], feed_dict={X_var:X, y_var:Y})
        print( 'train_loss: ' + str( train_loss ) )
        
        with tf.compat.v1.variable_scope("dekdee", reuse=tf.compat.v1.AUTO_REUSE):

            print( X_var.eval() )

sess.close()

input('...')

**[输出]:**您可以看到,在学习之前,值并不相等。

(10, 4)
tf.Tensor(
[[ 6.86645582e-02 -1.35832652e-01 -2.94875264e-01  2.37066336e-02]
 [ 5.62810563e-02 -8.80426466e-02 -3.90841156e-01  1.85440294e-02]
 [ 2.65050940e-02 -3.51844281e-02 -4.62463677e-01  8.80880933e-03]
 [ 1.03698615e-02 -1.23751881e-02 -5.21368802e-01  3.36679374e-03]
 [ 3.79285915e-03 -4.20535356e-03 -5.68293333e-01  1.14964601e-03]
 [ 1.35087373e-03 -1.41531334e-03 -6.05076790e-01  3.67792789e-04]
 [ 4.75763605e-04 -4.74975619e-04 -6.33896351e-01  1.12981070e-04]
 [ 1.66728903e-04 -1.59263654e-04 -6.56660855e-01  3.37953716e-05]
 [ 5.82966786e-05 -5.33882230e-05 -6.74850523e-01  9.92863897e-06]
 [ 2.03617565e-05 -1.78952469e-05 -6.89554811e-01  2.88080719e-06]], shape=(10, 4), dtype=float32)
...
train_loss: 46035130000.0
[[[ 0.74024   ]
  [-0.04634753]
  [-0.10950685]
  [-0.616641  ]
  [ 0.        ]
  [ 1.1847169 ]
  [-0.08368474]
  [-0.42080724]
  [-0.93031204]   👈
  [-0.6508291 ]]] 👈
train_loss: 46032654000.0
[[[ 0.7402304 ]
  [-0.04627251]
  [-0.10951507]
  [-0.6162482 ]
  [ 0.        ]
  [ 1.1846094 ]
  [-0.08343519]
  [-0.42065513]
  [-0.93027836]   👈
  [-0.65070885]]] 👈
train_loss: 46030190000.0
[[[ 0.74022084]
  [-0.04619753]
  [-0.10952328]
  [-0.61585563]
  [ 0.        ]
  [ 1.184502  ]
  [-0.0831857 ]
  [-0.4205031 ]
  [-0.93024474]   👈
  [-0.6505887 ]]] 👈

相关问题